Have Geek Brain

How to generate full URLs from incomplete URLs found on a web page?

I can get the text of a webpage, say this StackOverflow question with some real and generated links:

    / questions
    / tags
    / questions? sort = votes
    / questions? sort = active
    randompage.aspx
    ../coolhomepage.aspx

Knowing my original page was this StackOverflow question , is there a way in .Net to resolve links to this?

    / questions
    https://stackoverflow.com/tags
    / questions? sort = votes
    / questions? sort = active
    /questionsrandompage.aspx
    https://stackoverflow.com/coolhomepage.aspx

It's like the browser is smart enough to allow links.

================================================== ================================================== ================================================== ================================================== =============================================== 

    'Regex to match all <a ... /a> links
    Dim myRegEx As New Regex ("\ <\ s * a (? # Find opening <a tag)" & _
                             ". +? href \ s * = \ s * ['" "] (? # Then all to href =' or" ")" & _
                             "(? <href>. *?) ['" "] (? # Then all to the next' or" ")" & _
                             ". *? \> (? # Then all to>)" & _
                             "(? <name>. *?) \ <\ s * / a \ s * \> (? # Then all to </a>)", _
                             RegexOptions.IgnoreCase Or _
                             RegexOptions.IgnorePatternWhitespace Or _
                             RegexOptions.Multiline)

    'MatchCollection to hold all the links that are matched
    Dim myMatchCollection As MatchCollection
    myMatchCollection = myRegEx.Matches (Me._RawPageText)

    'Loop through all matches and evaluate the value of the href attribute.
    For i As Integer = 0 To myMatchCollection.Count - 1
        Dim thisLink As String = ""
        thisLink = myMatchCollection (i) .Groups ("href"). Value ()
        'This checks for Javascript and Mailto links.
        'This is not complete. There are others to check I just haven't encountered them yet.
        If thisLink.ToLower.StartsWith ("javascript") Then
            thisLink = "JAVASCRIPT:" & thisLink
        ElseIf thisLink.ToLower.StartsWith ("mailto") Then
            thisLink = "MAILTO:" & thisLink
        Else
            Dim baseUri As New Uri (Me.URL)

            If Not thisLink.ToLower.StartsWith ("http") Then
                'This is a partial URL so we will assume that it relative to our originating URL
                Dim myUri As New Uri (baseUri, thisLink)
                thisLink = "RELATIVE LOCAL LINK: RESOLVED:" & myUri.ToString () & "ORIGINAL:" & thisLink
            Else
                'The link starts with HTTP, determine if part of base host or is outside host.
                Dim ThisUri As New Uri (thisLink)
                If ThisUri.Host.ToLower = baseUri.Host.ToLower Then
                    thisLink = "INSIDE COMPLETE LINK:" & thisLink
                Else
                    thisLink = "OUTSIDE LINK:" & thisLink
                End If
            End If

        End If

        'I'm storing the found links into a Generic.List (Of String)
        'This link has descriptive text added to it.
        'TODO: Make collection to hold only unique internal links.
        Me._Links.Add (thisLink)
    Next
+1


a source to share


3 answers


Do you mean this?

Uri baseUri = new Uri("http://www.contoso.com");
Uri myUri = new Uri(baseUri, "catalog/shownew.htm");

Console.WriteLine(myUri.ToString());


The sample comes from http://msdn.microsoft.com/en-us/library/9hst1w91.aspx

+2


a source


If you mean server side, you can use ResolveUrl()

:



string url = ResolveUrl("~/questions");
+1


a source


I don't understand what you mean by "permission" in this context, but you can try to insert a basic html element. Since you asked how the browser will handle it.

"The tag <base>

specifies the default address or default target for all links on the page."

http://www.w3schools.com/TAGS/tag_base.asp

0


a source






More articles:


All Articles