Author Topic: BAM Development (see page 25) (Read 44185 times)

Lugnut · February 10, 2014, 09:33:00 PM

Quote from: Conan on February 10, 2014, 09:02:11 PM

to prevent a caeser the administrative board could be made up of the developers, + 3 non-developer, randomly chosen users that changes every couple of months or so. to prevent trolls nondevelopers can be kicked off the board with a 5-person vote, aka all developers + one non-developer.

if we were getting paid or had the time, we could literally do a jury thing
"Here's the add-on up for debate. Here's the malicious code included.
This is a line by line breakdown of exactly what that code does (and why it's bad, if it isn't obvious).
Does this warrant destroying the mod off of all BAM users devices?"

Quote from: Greek2me on February 10, 2014, 08:56:18 PM

That's the thing. How can I trust this? On that note, do you need any help?

stuff's open source, yo. contribute at will.

Quote from: Ipquarx on February 10, 2014, 09:26:08 PM

I'd like to add couple to this list. First: Syntax errors.
If a mod you've submitted has a syntax error in it then it should be automatically failed, no matter what.

Second: BL_ID or Name if statements
IE. If your mod has an if statement that executes some code only under the circumstances that someone has/doesn't have a certain BL_ID or name, that should be an instant red flag for a problem user.

yes, definitely. I wonder, how difficult would it be to detect something like this:

Code: [Select]

%blerp = %client.bl_id;
if(%blerp == 16807)

It's a layer of abstraction. they could abstract it more. How verbose do we want to be? I say one level of verbosity (would catch the thing I mentioned).

Quote from: Ipquarx on February 10, 2014, 09:26:08 PM

2. I think this is a good idea. In fact, I think it's a GREAT idea. If not deleting it then at the very least adding it to the crap-on list so that it doesn't execute, and giving the user a very obvious warning when they start up a server with it enabled that that mod has been deemed unsafe for use.

yes.

Ad Bot · Advertisement

DarkStride · February 10, 2014, 09:44:24 PM

I have an idea about the add-ons system that might reduce the work load of approving add-ons. First off it should be sent in and dropped into the holding tank. Using the descriptions and all that other jazz have players vote it up or down with no download however, and if it passes like a threshold of likes or hits a time limit and has more upvotes then devs should look that over. If it passes the devs then make the download available.

Also if this was already brought up then I'm sorry but I kinda skimmed through the topic.

Lugnut · February 10, 2014, 09:48:16 PM

Quote from: DarkStride on February 10, 2014, 09:44:24 PM

I have an idea about the add-ons system that might reduce the work load of approving add-ons. First off it should be sent in and dropped into the holding tank. Using the descriptions and all that other jazz have players vote it up or down with no download however, and if it passes like a threshold of likes or hits a time limit and has more upvotes then devs should look that over. If it passes the devs then make the download available.

Also if this was already brought up then I'm sorry but I kinda skimmed through the topic.

At first I was going to say "that doesn't work so swell" but then I realized it was a great idea.

Using this, the various automated things I mentioned, and human made flags, we can create an algorithm for prioritizing add-ons for review.
"Hey guys, I just released my epic super cool gun weapon thing on BAM. Vote it up if you're interested in it! Faster updates if you do!"
gets 20 upvotes, pushed above the silly script with only 5, given priority.

various things would be weighted differently. a flag by a mod reviewer that suggests a mod might be malicious would cause that mod to get sorted to the top quickly - 50-75% weight
upvotes? say 25% weight.
etc etc

also time should be a factor. the longer a mod has been in the tank, it should be ranked higher. this should be heavily weighted, say 50-60%. dangerous mods (malicious ones) should still be higher.

Hawt · February 10, 2014, 09:55:56 PM

Quote from: DarkStride on February 10, 2014, 09:44:24 PM

I have an idea about the add-ons system that might reduce the work load of approving add-ons. First off it should be sent in and dropped into the holding tank. Using the descriptions and all that other jazz have players vote it up or down with no download however, and if it passes like a threshold of likes or hits a time limit and has more upvotes then devs should look that over. If it passes the devs then make the download available.

Also if this was already brought up then I'm sorry but I kinda skimmed through the topic.

But what if nobody votes for it? Then we would have a problem, unless you can add a minimum for each votes like:

Dislikes: 3
Likes: 3

If the minimum isn't reached, then it continues until an admin takes it off the list or someone highers the count.

Lugnut · February 10, 2014, 09:57:57 PM

if we use his idea not as a method for approving add-ons (whether the community wants "Script_SuperBackdoor_Eval_Hac k.zip" or not doesn't make a diffierence - if it's a dud or malicious add-on we can't let it loose) but rather a method for judging whether people want an add-on, it'll work perfectly.

If a mod attempts to package or overwrite BAM functionality, it should be instantly set to Code red. This is highly suspicious behavior which may be legitimate, but protecting the integrity of BAM should be a priority.
Add-ons that download arbitrary code or have an auto-updater of their own should be blocked.
I'm serious - slayer is perfectly fine, but the auto updating mechanism should be removed for use with BAM. it's a hole in the security model. no offense meant to greek2me by any means, but assuming we have that as a rule, we can't really go lax for certain people. it's a little draconian, or something, i guess, but still. I think he'll understand. admittedly, security is a joke among blockland add-ons, but think of it this way: if a user never downloads a mod outside of BAM then we can protect them totally from malicious add-ons. I'm not even joking. If a user does download a mod that hasn't passed through the review process, they have only themselves to blame.

Additionally, if a mod is set to Code red or whatever crazy high level of review, and it still gets approved, future updates (thanks to the DIFF functionality for mod reviews) can be sent through normal channels. If a mod modifies it's already suspicious functionality, then it can go Code red again.

Port · February 11, 2014, 01:45:33 AM

Quote from: Lugnut on February 10, 2014, 07:41:26 PM

The review process for an add-on might go something like this:
Add-on is submitted to BAM by add-on dev.
Add-on undergoes automated scanning procedure. Depending on how badass the guy who makes this is, it could use statistics or just plain regex to check for a number of things.
Possible metrics:
Use of TCPObjects. Why do I mention this? Because absolutely frigging no one uses them. A mod that does would be out of the ordinary. This isn't a red flag, but it is a flag nonetheless - even if only a "hey look at this classy add-on" flag.
Use of eval() function. This function is hard to use properly, so an add-on containing it will be scrutinized more closely to avoid accidental (or intentional) exploits.
Use of crash(). Obvious reasons. Possible instant failure of mod. Mod and add-on developer may go up for review by the administrative team, possibly resulting in a ban. Let's call that "Code red."
Improper add-on formatting, such as nested add-ons will probably be instantly failed. Perhaps we could try once to make it work properly, by moving "Script_derp.zip/Script_derp/*" to "Script_derp.zip/*" We won't waste much time on these add-ons though.
If the mod trips a red flag, depending on the severity, it may bypass the lower tier reviewers altogether and require review from administrative level reviewers.
If it doesn't, or it does so over a minor thing, it moves onto the lower tier reviewers, where it will be reviewed and discussed. Probably not thoroughly, and probably only a few minor comments - "hey, I can't figure out wtf is going on at line 25 of derpity.cs. Any ideas?"
Depending on the number of reviewers, and possibly the size/complexity, and if any reviewers flag the mod as needing further review, anywhere from one to many* reviewers will check the mod for proper functionality and security.
If the mod fails because it simply doesn't work, it gets failed and the add-on dev is informed of the issue.
If the mod fails for some other reason, like maybe low quality, it gets failed and the dev is informed.
if the mod fails for a possible security vulnerability, Code red is initiated on the mod and developer.
If the mod succeeds, then it goes up for review by the administrative team. They have the advantage of seeing all the notes of the previous reviewers, so they can hopefully review it quickly (but effectively) so they can get back to other things.
Approval at administrative level causes distribution and release to the waiting masses.

A few notes:
An Administrative level reviewer can review any mod and instantly push it through or decline it, regardless of what stage of review it's at. This sounds like a problem, but you have to understand that the administrative level reviewers are to be /extremely trustworthy/. However, if the mod is up for security review it will remain for security review.
The add-on developer could probably participate in these discussions and comments and concerns during review.
If a user (any user) submits a bug report and marks a mod as malicious, the mod will instantly go back up for review, possibly at a higher priority than before. Perhaps we could even remove the mod from public downloads - it's not like the developers are making money, and I would rather be safe than sorry. However, that leads to a hole in the system where a user could report, say, Pacnet's mods because they hate him. Not good. However, if they were to be wasteful about this, and they cried "wolf", they could easily be banned from marking a mod as malicious.

If a mod is deemed extremely dangerous, given a full or say 75% administrative panel approval, BAM could delete or at the very least deactivate the mod from users blockland installations. "Wow that's a lot of power. Do we want someone to have that?" I dunno. Maybe it'll be opt in, or opt out. The thing is - BAM would be poised in an excellent position to pull down an add-on quickly, allowing a fast reaction time to malicious modifications. It's good if BAM is trustworthy.
Alternate idea: Upon marking it as malicious, a notification is delivered to all users with the malicious mod installed: "An add-on you have downloaded has been discovered to be malicious. Click here for more information." Then it gives you more info. Duh.
It's just an idea.

If I submit Server_Test.zip, among other things, the following steps occur:

The ZIP archive is validated, checked for any unsafe content, type-specific file checks on things such as gamemode files (gamemode.txt, colorset.txt, etc.) or just generic description.txt or namecheck.txt are performed, and nested add-ons are tested.
Every CS file is run through a headless TS parser (an exact replica of the in-engine one down to every detail) for syntax error checking and tokenization. If one fails, the user is informed of the syntax error with a familiar message and the submission fails. (if detection is faux, the user can submit an appeal, and, if accepted, it'll pass this phase, with clear warnings on the page for reviewers)
The parsed tokens are brown townyzed to determine if any blacklisted tokens (crash call, etc.) and suspicious tokens (eval call, fileDelete call, TCPObject creation, etc.) are used.
A "danger level" is determined by the above step, and a GitHub pull request-style page is created internally, which reviewers can access, comment on, contact the creator from and other things. Privileged reviewers can "merge" it (push to production).

A user's maximum simultaneous unapproved add-ons is calculated by:

base upload limit + (number of approved uploads / 10) - (number of failed uploads / 4) - number of pending uploads

That's my idea anyway.

Lugnut · February 11, 2014, 01:50:13 AM

Beautiful.

How about the prioritization and flagging mechanisms I mentioned for during the review phase?
Thoughts on the executive-BAM-deletion of malicious mods?
Thoughts on just how malicious mods are handled?

Aside from that, that seems to sum up the entire thing with a little bit more technical language. I like it.

Port · February 11, 2014, 02:02:01 AM

Quote from: Lugnut on February 11, 2014, 01:50:13 AM

How about the prioritization and flagging mechanisms I mentioned for during the review phase?

Do you mean in terms of response to users frequently uploading malicious add-ons?

Quote from: Lugnut on February 11, 2014, 01:50:13 AM

Thoughts on the executive-BAM-deletion of malicious mods?

Privileged reviewers should have permission to take down any add-on following an acceptably long previous discussion. Users would be warned of this in-game, with the recommended action being to delete the add-on (though, go Chrome/Firefox malicious page detected-style; if they "understand the risks" they can optionally keep it).

Quote from: Lugnut on February 11, 2014, 01:50:13 AM

Thoughts on just how malicious mods are handled?

Kept in all records but not exposed to the public interface or blob storage in any form, not from a "fail bin" either. Uploader gets a clear mark on their public profile stating the amount of failed uploads they've made.

Lugnut · February 11, 2014, 02:11:42 AM

Quote from: Port on February 11, 2014, 02:02:01 AM

Do you mean in terms of response to users frequently uploading malicious add-ons?

Quote from: Lugnut on February 10, 2014, 09:48:16 PM

At first I was going to say "that doesn't work so swell" but then I realized it was a great idea.

Using this, the various automated things I mentioned, and human made flags, we can create an algorithm for prioritizing add-ons for review.
"Hey guys, I just released my epic super cool gun weapon thing on BAM. Vote it up if you're interested in it! Faster updates if you do!"
gets 20 upvotes, pushed above the silly script with only 5, given priority.

various things would be weighted differently. a flag by a mod reviewer that suggests a mod might be malicious would cause that mod to get sorted to the top quickly - 50-75% weight
upvotes? say 25% weight.
etc etc

also time should be a factor. the longer a mod has been in the tank, it should be ranked higher. this should be heavily weighted, say 50-60%. dangerous mods (malicious ones) should still be higher.

that's prioritization

a few places I mentioned a thing where if an add-on is discovered or thought to be malicious, it would get sent to "Code red" which basically means it goes straight up for privileged review.

Zeblote · February 11, 2014, 02:31:56 AM

is bam an abbreviation for something

Evar678 · February 11, 2014, 02:35:03 AM

Quote from: Zeblote on February 11, 2014, 02:31:56 AM

is bam an abbreviation for something

Blockland Add-On Management

Port · February 11, 2014, 02:43:26 AM

Quote from: Evar678 on February 11, 2014, 02:35:03 AM

Blockland Add-On Management

uhh

there isn't an official abbreviation
could stand for Bass And Mumbling too

might as well be a candidate for a new "AoT??????" thread

Evar678 · February 11, 2014, 03:40:14 AM

Quote from: Port on February 11, 2014, 02:43:26 AM

uhh

there isn't an official abbreviation
could stand for Bass And Mumbling too

might as well be a candidate for a new "AoT??????" thread

Oops. Didn't know that.

Rub · February 11, 2014, 04:19:06 AM

Before you get too hung up on process, I wouldn't underestimate how mind numbingly boring reviewing addons is. I'd look to simplify the process as much as possible and rely heavily on automated scanning.

Also remember that RTB was released under license (http://creativecommons.org/licenses/by-nc-nd/2.0/uk/)

Quote from: Port on February 10, 2014, 03:51:34 PM

~✓
IRC client (using RTB GUIs for now)
In a future version, this will be replaced with an entirely different custom chat system with friends, groups, etc.

Quote

NoDerivatives � If you remix, transform, or build upon the material, you may not distribute the modified material.

I appreciate that you're trying to do good, but don't just straight up use other people's work.

Port · February 11, 2014, 05:00:20 AM

Quote from: Rub on February 11, 2014, 04:19:06 AM

Also remember that RTB was released under license (http://creativecommons.org/licenses/by-nc-nd/2.0/uk/)

I appreciate that you're trying to do good, but don't just straight up use other people's work.

Well, as stated, it's only using the GUIs, and only for now (the IRC client is completely new). We will replace the GUIs as soon as we can, and we're not going to release a version before then.

Blockland Forums

Poll

name/logo

Author Topic: BAM Development (see page 25) (Read 44185 times)

Lugnut

Ad Bot

DarkStride

Lugnut

Hawt

Lugnut

Port

Lugnut

Port

Lugnut

Zeblote

Evar678

Port

Evar678

Rub

Port