Grab URLs

Questions and information about creating Internet aware NeoBook applications. Including PHP, HTML, FTP, HTTP, Email, etc.

Moderator: Neosoft Support

Locked
User avatar
stu
Posts: 322
Joined: Wed Aug 07, 2013 11:37 am

Grab URLs

Post by stu »

Any plugin or magic that would allow to get a list of urls from a text. Like InternetGet "URL" "[HTML]"

And then somehow I get a list of all urls within the var?

Cheers.
User avatar
dec
Posts: 1663
Joined: Wed Nov 16, 2005 12:48 am
Location: Spain
Contact:

Re: Grab URLs

Post by dec »

Hello,

Maybe the use of Regular Expression can be useful here. You can try my npRexp plugin.

Anyway it's not an easy thing, as you can see, then may you can be a little more specific... ;)
.
Enhance your NeoBook applications!
.
58 plugins, 1131 actions and 233 samples
.
NeoPlugins website: www.neoplugins.com
.
User avatar
stu
Posts: 322
Joined: Wed Aug 07, 2013 11:37 am

Re: Grab URLs

Post by stu »

I think the Techmedia dude had an URL scanner that would get all JPGs, JSs, etc from a designated URL... Of course, his out of the picture so I was wondering if there's any alternative way of getting links and such from a text... but yeah, maybe regural expressions would aid... I'll check it out. :D
David de Argentina
Posts: 1596
Joined: Mon Apr 04, 2005 4:13 pm
Location: Buenos Aires, Argentina
Contact:

Re: Grab URLs

Post by David de Argentina »

Hi stu,

You could addapt my NeoThinBasic sample for extract any image from a .EML file.
In fact, an .EML is almost equal an .HTML file.
Take a look of this:
viewtopic.php?f=34&t=20213

My devaluated cent,
David de Argentina
User avatar
dec
Posts: 1663
Joined: Wed Nov 16, 2005 12:48 am
Location: Spain
Contact:

Re: Grab URLs

Post by dec »

Hello,

Take a look at the below sample:

http://www.neoplugins.com/Humm/Sites/Ma ... abURLs.zip

Instead of searching for URLs, I use a Regular Expression to search any "href" or "src" URL.
.
Enhance your NeoBook applications!
.
58 plugins, 1131 actions and 233 samples
.
NeoPlugins website: www.neoplugins.com
.
User avatar
stu
Posts: 322
Joined: Wed Aug 07, 2013 11:37 am

Re: Grab URLs

Post by stu »

That's very nice, only one button... Thanks for the example, I'll take a closer look at npRexp. =)
User avatar
dec
Posts: 1663
Joined: Wed Nov 16, 2005 12:48 am
Location: Spain
Contact:

Re: Grab URLs

Post by dec »

Hi!
stu wrote:That's very nice, only one button... Thanks for the example, I'll take a closer look at npRexp. =)
You're welcome stu!
.
Enhance your NeoBook applications!
.
58 plugins, 1131 actions and 233 samples
.
NeoPlugins website: www.neoplugins.com
.
User avatar
stu
Posts: 322
Joined: Wed Aug 07, 2013 11:37 am

Re: Grab URLs

Post by stu »

Hello! =)

I was trying to use the example to grab some URLs from Google search through the custom search API but im not getting postive results, any idea? I can clearly see URLs there and I try with other sites and works fine (?)

check:

Code: Select all

https://www.googleapis.com/customsearch/v1?key=AIzaSyBFD2lX-rzYdCYNabpCMnL5xuHgWJpkRHg&cx=017576662512468239146:omuauf_lfve&q=lectures
:wink:
User avatar
dec
Posts: 1663
Joined: Wed Nov 16, 2005 12:48 am
Location: Spain
Contact:

Re: Grab URLs

Post by dec »

Hello,

The above sample it's based in the "href" and "src" HTML tags attributes. However, the link you shown:

Code: Select all

https://www.googleapis.com/customsearch/v1?key=AIzaSyBFD2lX-rzYdCYNabpCMnL5xuHgWJpkRHg&cx=017576662512468239146:omuauf_lfve&q=lectures
... Do not offer HTML source, but JSON, if I am not wrong. Then yes, we can view here some URLs, but not in the "href" or "src" HTML tags, because there is no HTML tags.

If you take a look at this other sample:

http://www.neoplugins.com/Humm/Sites/Ma ... bURLs2.zip

What I do is to change the Regular Expression in order to grab any URL in the source JSON "link:" property, something like:

Code: Select all

npRexpStr "[ID]" "[#34]link[#34]: [#34](.*)[#34]" "[Result]"
And now yes... you can view how we can grab such URLs from the above JSON source.
.
Enhance your NeoBook applications!
.
58 plugins, 1131 actions and 233 samples
.
NeoPlugins website: www.neoplugins.com
.
User avatar
fkapnist
Posts: 348
Joined: Mon Nov 17, 2014 4:24 pm
Location: Greece
Contact:

Re: Grab URLs

Post by fkapnist »

Put something like this in your JS Functions folder and Call to it.... It writes a list to a IE window but you can modify it to save as a file, etc...
var allLinks = document.links;
for (var i=0; i<allLinks.length; i++) {
win.document.write(allLinks.href+"<BR/>"); // this works


8)

.
Locked