Introduction:
There is a lot of information available in the internet like Breaking News, Sports News, and Financial Information’s, Business rules, Stock exchange information and lot of. If any one needs information he just type google.com and start searching. After he finds his informational page then he just navigates to this page and extracts his related information. Developers also need to do these using programming languages to find latest information’s in the internet. The technique developer’s use is the process of extracting data from the HTML is called screen scraping because it's scraping the data off the screen instead of getting the data more directly. In this Article you will see how we create dynamic Page instance which contains the information of any URL.
Description:
When I reply to this post I then I decided to write an article for this. The logic behind this article is same that was used previously with Screen Scraping i.e construct a request, parse the response. Parsing the HTML is often the trickiest part of the problem. You may use Regular Expression to get your desire result from HTML response. But this need lot of work and need to know the knowledge of Regex and may also not give 100% result. Therefore for need a simple Object Oriented way is that just use VirtualPathProvider class provided by .NET Framework.
VirtualPathProvider allows you to implement some sort of “virtual URL” accessible on the server. This gives you the possibility of generating a response for a URL dynamically without having an ASPX or HTML file stored on the hard disk. Therefore First of all you just needs to create and add a new class to the App_Code directory and inherit from VirtualPathProvider.
public class MyProvider : System.Web.Hosting.VirtualPathProvider
{
public static void AppInitialize()
{
MyProvider fileProvider = new MyProvider(); System.Web.Hosting.HostingEnvironment.RegisterVirtualPathProvider(fileProvide)
}
public override bool FileExists(string virtualPath)
{
if (virtualPath.EndsWith("YourVirtualPath.aspx"))
return true;
else
return Previous.FileExists(virtualPath);
}
public override System.Web.Hosting.VirtualFile GetFile(string virtualPath)
{
if (virtualPath.EndsWith("YourVirtualPath.aspx"))
{
string s = "http://www.bing.com/search?q=Searching+Text+Box+In+Bing+Site&go=&form=QBRE&qs=n";
if (Context.Items["url"]!=null)
s=Context.Items["url"].ToString();
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(s);
HttpWebResponse respons;
respons = (HttpWebResponse)request.GetResponse();
StreamReader reader = new StreamReader(respons.GetResponseStream());
string contents = reader.ReadToEnd().Replace("<input","<input runat=\"server\"");
return new MyVirtualFile(virtualPath, contents);
}
else
return Previous.GetFile(virtualPath);
}
}
The main point is to note here is the AppInitialize method,
public static void AppInitialize()
{
MyProvider fileProvider = new MyProvider(); System.Web.Hosting.HostingEnvironment.RegisterVirtualPathProvider(fileProvide) }
If this method is present in a VirtualPathProvider class; it is automatically called by the framework which creates an instance of this class and registers itself with framework.
Next, you must sure that you have not any YourVirtualPath.aspx present physically because it will never call.
if (virtualPath.EndsWith("YourVirtualPath.aspx"))
Another important Point note here is that,
string contents = reader.ReadToEnd().Replace("<input","<input runat=\"server\"");
Here I only replace <input with <input runat="server", i.e., it will only includes controls in Page class which are input controls. Therefore if you need other element to show in Control Collocation (not only input) of page, do it your self.
For Example for a Label,
Just replace <span to <span runat="server"
The Other details of VirtualPathProvider class you get from here.
You have also needs to create your own implementation of VirtualFile and override the Open method. This method then returns a stream to the ASP.NET infrastructure, which actually returns the contents. You also place this class in App_Code Folder.
public class MyVirtualFile : System.Web.Hosting.VirtualFile
{
private string _FileContent;
public MyVirtualFile(string virtualPath, string fileContent)
: base(virtualPath)
{
_FileContent = fileContent;
}
public override Stream Open()
{
Stream stream = new MemoryStream();
StreamWriter writer = new StreamWriter(stream, Encoding.Unicode);
writer.Write(_FileContent);
writer.Flush();
stream.Seek(0, SeekOrigin.Begin);
return stream;
}
}
Finally you need to dynamically create a page instance in your code, like
protected void Page_Init(object sender, EventArgs e)
{
Context.Items["url"] = "http://www.bing.com/search?q=Searching+Text+Box+In+Bing+Site&go=&form=QBRE&qs=n";
Page myPage = (Page)BuildManager.CreateInstanceFromVirtualPath("~/YourVirtualPath.aspx", typeof(Page));
myPage.ProcessRequest(HttpContext.Current);
Response.ClearContent();
Response.ContentType = "text/html";
You must clear all the Content that is added to this current Page. Here you can find your control use FindControl method.
((HtmlInputText)myPage.FindControl("TextBox1")).Value
Summary:
I think you will find some thing interesting here with VirtualPathProvider class. However the VirtualPathProvider class gives you the additional possibility of deploying your web application (or, rather, parts of your web application). Actually, you have the ability to store “pages” of the web application somewhere other than on the file system without writing your own basic page framework that uses information in the database for dynamically creating controls and adding them to the page. You can just retrieve the whole file from the database and pass it to the ASP.NET runtime for further processing. The runtime treats the information retrieved from the database (or any other data store) like a physical page located on the file system. And that’s not all—you can use the VirtualPathProvider class for accessing other features, such as themes and skins, from a different location than the file system.