Grab URLs

[stu]

Any plugin or magic that would allow to get a list of urls from a text. Like InternetGet "URL" "[HTML]"

And then somehow I get a list of all urls within the var?

Cheers.

[dec]

Hello,

Maybe the use of Regular Expression can be useful here. You can try my npRexp plugin.

Anyway it's not an easy thing, as you can see, then may you can be a little more specific...

[stu]

I think the Techmedia dude had an URL scanner that would get all JPGs, JSs, etc from a designated URL... Of course, his out of the picture so I was wondering if there's any alternative way of getting links and such from a text... but yeah, maybe regural expressions would aid... I'll check it out.

[David de Argentina]

Hi stu,

You could addapt my NeoThinBasic sample for extract any image from a .EML file.
In fact, an .EML is almost equal an .HTML file.
Take a look of this:
viewtopic.php?f=34&t=20213

My devaluated cent,
David de Argentina

[dec]

Hello,

Take a look at the below sample:

http://www.neoplugins.com/Humm/Sites/Ma ... abURLs.zip

Instead of searching for URLs, I use a Regular Expression to search any "href" or "src" URL.

[stu]

That's very nice, only one button... Thanks for the example, I'll take a closer look at npRexp. =)

[dec]

Hi!

That's very nice, only one button... Thanks for the example, I'll take a closer look at npRexp. =)

You're welcome stu!

[stu]

Hello! =)

I was trying to use the example to grab some URLs from Google search through the custom search API but im not getting postive results, any idea? I can clearly see URLs there and I try with other sites and works fine (?)

check:

Code: Select all

https://www.googleapis.com/customsearch/v1?key=AIzaSyBFD2lX-rzYdCYNabpCMnL5xuHgWJpkRHg&cx=017576662512468239146:omuauf_lfve&q=lectures

[dec]

Hello,

The above sample it's based in the "href" and "src" HTML tags attributes. However, the link you shown:

Code: Select all

https://www.googleapis.com/customsearch/v1?key=AIzaSyBFD2lX-rzYdCYNabpCMnL5xuHgWJpkRHg&cx=017576662512468239146:omuauf_lfve&q=lectures

... Do not offer HTML source, but JSON, if I am not wrong. Then yes, we can view here some URLs, but not in the "href" or "src" HTML tags, because there is no HTML tags.

If you take a look at this other sample:

http://www.neoplugins.com/Humm/Sites/Ma ... bURLs2.zip

What I do is to change the Regular Expression in order to grab any URL in the source JSON "link:" property, something like:

Code: Select all

npRexpStr "[ID]" "[#34]link[#34]: [#34](.*)[#34]" "[Result]"

And now yes... you can view how we can grab such URLs from the above JSON source.

[fkapnist]

Put something like this in your JS Functions folder and Call to it.... It writes a list to a IE window but you can modify it to save as a file, etc...

var allLinks = document.links;
for (var i=0; i<allLinks.length; i++) {
win.document.write(allLinks.href+"<BR/>"); // this works

.