Is there a way to log forum profiles? [Solved]

[Kyuande]

Is there a way to log forum profile name/bl_id/profileID without doing an endless loop of profile IDs with a TCP object?

[Ad Bot]

[phflack]

on the bright side
you don't need an endless loop since the newest is u=145883
and so you can count up/down to there

also, could go through the member list http://forum.blockland.us/index.php?action=mlist

[Kyuande]

Thanks! I'll start the loop from there then by using onDisconnect for the next search.

I'm not sure if this would count as spamming the forums with tcp objects to scan for profiles with existing bl_ids in a 0.5 second timeout per request. It should only do this once and then if a player connects with a bl_id as the same as the profile it'll update some information if it can.

This will be an interesting project I am going to do.

[jes00]

Thanks! I'll start the loop from there then by using onDisconnect for the next search.

I'm not sure if this would count as spamming the forums with tcp objects to scan for profiles with existing bl_ids in a 0.5 second timeout per request. It should only do this once and then if a player connects with a bl_id as the same as the profile it'll update some information if it can.

This will be an interesting project I am going to do.
[img]http://i.imgur.com/bgl90dc.png[/img]

Motto?

EDIT: Never mind. It's the personal text.

[$trinick]

http://forum.blockland.us/index.php?action=mlist will cut down your requests 30 fold, but only will let you link URL/forum ID to name.

[Zeblote]

Checking all those profiles once would be half the usual page views the forum gets in a day. If this lookup is really necessary, you should probably download everything once and store blid->forum id locally to query it much faster.

Also, your IP will probably get blocked by cloudflare in the process.

[Kyuande]

It will just request the ID after it disconnects, it shouldn't be blocked. It does store it in a file, as I've gotten up to almost 10k of the profile ID for today.
It does not store profiles if they don't have a BL_ID.

It will store what it shows in the console, here's an example:
$ForumProfiler::Profile0 = "2"; //ProfileID for the BL_ID
$ForumProfiler::Profile0_LastActive = "Today at 12:36:09 AM"; //Last active
$ForumProfiler::Profile0_Motto = ""; //Personal text
$ForumProfiler::Profile0_Name = "Badspot"; //Name on the profile
$ForumProfiler::Profile0_Posts = "7768 (2.021 per day)"; //Post count
$ForumProfiler::Profile0_Registered = "March 07, 2005, 03:12:46 AM"; //Date of registration

I'm not sure if I should keep going with this project, the requests are slow so they should not attack the forums.
Problem is when it is done, may lag the client after it execs once. The file below shows all I've recorded that has BL_IDs.

[Dannu]

Depending on the intended usage, you could just have a lookup table of sorts. If you want to reference by multiple factors, have a per-client file with all the info and a .txt file with like the name or BL_ID as the name of the file, that references to the info file.

[Kyuande]

The basic usage is when I want to look someone up I can see if they are on the forums. May not sound useful at the moment, but it can create a database. I might just stop this anyways because I'm not sure what technical use is for this.

[Dannu]

Then just make the profiles themselves with the name of the player's BLID.

[Lugnut]

Protip: use this list
http://forum.blockland.us/index.php?action=mlist;sort=posts;start=%num
go from 0 to 16110, extract the profile links from those pages
http://forum.blockland.us/index.php?action=mlist;sort=posts;start=16110

those are the only ones you really need to download, as it isn't extremely likely that someone would have tied a blid to an account with zero posts
possible. almost positively happened. but isn't very likely. furthermore, if they've made posts since after blid's became mandatory for posting, then they /have/ to have a blid to post... so the bulk of these people will probably have blids attached to their posts

16110 queries isn't nearly as bad as 145883.

also, if you do go to the trouble of getting all these profiles downloaded, please post the database somewhere in a zip file so other modders can use it

[Kyuande]

Yeah, this should save me a lot of time instead of doing loops of 1 by 1, which creates 404 issues, good thing it only logs accounts with existing bl_ids and IDs. Thanks.

[ZSNO]

You may also be able to speed your process up a lot by using this list hammereditor made a long time ago.
http://www.hammereditor.net/files/bl_id_index.txt

[Kyuande]

I've looked at that before but I wasn't sure if I should implement it. I should.

[Swollow]

it shouldn't be blocked.

you have a variable chance of getting blocked especially if you are doing one request after another
trust me I've done a lot of scraping on blf