playing with the mast0rlist

wurst · **Posted:** 09.03.12

He, im just sitting here and think about wether my server is on the masterlist.
After thinking some time more, i have this as a result:

Code:

<?php

$version = $_GET['v'];

if ($version == '42') {$master_udp_string = "\xFF\xFF\xFF\xFFgetservers 70 full empty\x00";}
elseif ($version == '41') {$master_udp_string = "\xFF\xFF\xFF\xFFgetservers 68 full empty\x00";}
else {echo "please use download.php?v=42 or download.php?v=41"; die;}

function hex2tostr($hex)
{
$str='';
for ($i=0; $i < strlen($hex)-1; $i+=2)
{
if ($i != 0) {$str .= '.';}
$str .= (hexdec($hex[$i].$hex[$i+1]));
}
return $str;
}

function hex4tostr($hex)
{
$str='';
for ($i=0; $i < strlen($hex)-1; $i+=4)
{
$str .= (hexdec($hex[$i].$hex[$i+1].$hex[$i+2].$hex[$i+3]));
}
return $str;
}

$fp = fsockopen('udp://91.121.174.74', 27950, $errno, $error, 1);
fwrite($fp, $master_udp_string);
$read = fread($fp, 100000);
$s = substr($read, "4");

$split = explode("\\",$read);

// echo masterlist
$i = count($split);

while($i > 0) {

$hexip = substr("$split[$i]",0,4);
$ip = hex2tostr(bin2hex($hexip));
$hexport = substr("$split[$i]",4,5);
$port = hex4tostr(bin2hex($hexport));

if ($ip != ''){
echo "$ip:$port ";
}
$first = 0;
$i--;
}

?>

Thats teh output:

dswp_tools/masterlist/download.php?v=41
dswp_tools/masterlist/download.php?v=42

so u might say now: cool!

however, theres a problem.
masterlist is sending me multiple UDP packets as an answer, the php just take 1 first packet from the response.
i know that fread() would limit the output, however checked that already...

wurst · **Posted:** 09.03.12

OKi here we go...

Code:

<?php

$version = $_GET['v'];

if ($version == '42') {$master_udp_string = "\xFF\xFF\xFF\xFFgetservers 70 full empty\x00";}
elseif ($version == '41') {$master_udp_string = "\xFF\xFF\xFF\xFFgetservers 68 full empty\x00";}
else {echo "please use download.php?v=42 or download.php?v=41"; die;}

function hex2tostr($hex)
{
$str='';
for ($i=0; $i < strlen($hex)-1; $i+=2)
{
if ($i != 0) {$str .= '.';}
$str .= (hexdec($hex[$i].$hex[$i+1]));
}
return $str;
}

function hex4tostr($hex)
{
$str='';
for ($i=0; $i < strlen($hex)-1; $i+=4)
{
$str .= (hexdec($hex[$i].$hex[$i+1].$hex[$i+2].$hex[$i+3]));
}
return $str;
}

$fp = fsockopen('udp://91.121.174.74', 27950, $errno, $error, 1);
@socket_set_timeout($fp, 1);
fwrite($fp, $master_udp_string);

$test = $byte = fread($fp, 1);
while($test == chr(255)) {
$socket_status = socket_get_status($fp);
$byte_collect .= $test;
while($socket_status['unread_bytes'] > 0) {
//echo $socket_status['unread_bytes']; echo " ";

$count++;
$byte_collect .= $byte = fread($fp, 1);
$socket_status = socket_get_status($fp);
}
socket_set_blocking($fp, 0);
$test = fread($fp, 1);
socket_set_blocking($fp, 1);
}

// echo $byte_collect;

$split = explode("\\",$byte_collect);

// echo masterlist
$i = count($split);

while($i > 0) {

$hexip = substr("$split[$i]",0,4);
$ip = hex2tostr(bin2hex($hexip));
$hexport = substr("$split[$i]",4,5);
$port = hex4tostr(bin2hex($hexport));

if ($ip != '' && $port != ''){
echo "$ip:$port ";
}
$first = 0;
$i--;
}

?>

dswp_tools/masterlist/download.php?v=41
dswp_tools/masterlist/download.php?v=42

Result: 693 x Urt 4.11 vs. 201 Urt 4.2 servers 1 month after launch.

This helped me out:
http://www.php-resource.de/forum/php-de ... uffer.html

Samtron · **Posted:** 09.04.12

Can you also read out the amount of players? Would be interesting if there are also less 4.2 players than on 4.1.1 servers...

Unclefragger · **Posted:** 09.04.12

4.2 is officially released now? 8o
EDITH: just checked, 4.2 Beta is the "official" UrT now, but still beta ... =/

bky · **Posted:** 09.04.12

BrainBug from php-resource.de wrote:

sorry, nein, zu deinem Perl-Thema kann ich dir leider nicht weiterhelfen. Kenne mich mit Perl so gut wie gar nicht aus.

wursti hacking, nice :mrgreen:

wurst · **Posted:** 09.04.12

why not hack urself today?

@no-of-player-per-server-question:

what the master server answers is just server addresses.
no name, slots, map etc.
however, if u look a bit to the right (or on top)
the scripts with that we monitor our servers speak "normal" rcon to the gameservers.
sure we can do so with any server, in theory.
practically this means working out a fancy concept that goes ok with 16GB Ram and 1-2 SSD.

as such a "gametrackish thingy" is missing for URT in separate versions, it could mean some load on the web interface.
specially cause i say no to advertisements, it should be a clean and fast HTML output.

@features/fancy concept:
- to query the masterlist is NOT resource intensive, its recieving 2-3-4 Packets and making dec from hex from bin.

-querying ~800-900 Servers from both lists same time means eating RAM.
If the script fires up inside a heavy apache2 1000 times: omg!
maybe it could make sense to have it in c, ill c.

-storing everything in mysql shouldnt be that problem, even in narrow monitoring cycles like 1 minute.
precision should be lowered over the time in datastore.
like storing 1 month every minute, 1 month - 1 yr ago every hour, etc.
output must be fucking fast, keeping database layout simple and flexible would be 1 important goal.

Additional Features that i miss on gametracker etc:
--> id love to have a green "this server is currently in masterlist and reachable"- button
--> non-gaming related tests like icmp ping/traceroute, reverse DNS.
--> maybe some attack tests like UDP rddos, q3infoboom et al, registration should be required, some extra work again.

What i wouldnt try:
collecting playerstats. sry, what the gametracker brings occour poor to me. when i compare with our beloved b3-truth.
I mean: RastaSkud has 171958 Kills. 171958? Really? i count 417166.
they counted 41% of his kills, 41,22052132724143 to be precise. precise? ah, right, there was something that says me: *u waste ur storage*.
players change their names, nobody will stop them.
i wouldnt be satisfied when i watch those numbers.
they would multiplicate the amount of stored data several times.
i dont wanna try what the gametracker does pretty well.

eXtr33m · **Posted:** 09.04.12

Got the sockets working... looks bit cleaner than rading by byte

Modified some snippet from php.net and wursti's code.

Code:

<?php
if (!isset($_GET['v'])) die('Version not set.');
$version = intval($_GET['v']);
if ($version == '42') {$master_udp_string = "\xFF\xFF\xFF\xFFgetservers 70 full empty\x00";}
elseif ($version == '41') {$master_udp_string = "\xFF\xFF\xFF\xFFgetservers 68 full empty\x00";}
else {echo "please use download.php?v=42 or download.php?v=41"; die;}
function hex2tostr($hex) {
$str='';
for ($i=0; $i < strlen($hex)-1; $i+=2) {
if ($i != 0) {$str .= '.';}
$str .= (hexdec(substr($hex,$i,2)));
}
return $str;
}
function hex4tostr($hex) {
$str='';
for ($i=0; $i < strlen($hex)-1; $i+=4) {
$str .= (hexdec(substr($hex,$i,4)));
}
return $str;
}
if (!extension_loaded('sockets')) {
die('The sockets extension is not loaded.');
}
// create socket
$socket = socket_create(AF_INET, SOCK_DGRAM, getprotobyname("udp"));
if (!$socket) die('Unable to create AF_INET socket');
// connect
$address = '91.121.174.74';
$address_port = 27950;
socket_connect($socket, $address, $address_port);
$msg = $master_udp_string;
$len = strlen($msg);
// at this point 'server' process must be running and bound to receive from serv.sock
$bytes_sent = socket_sendto($socket, $msg, $len, 0, $address, $address_port);
if ($bytes_sent == -1) die('An error occured while sending to the socket');
else if ($bytes_sent != $len) die($bytes_sent . ' bytes have been sent instead of the ' . $len . ' bytes expected');
$tbuf = '';
$buf = '';
$totalBytes = 0;
$bytes_received = 0;
// set timeout
socket_set_option($socket, SOL_SOCKET, SO_RCVTIMEO, array("sec" => 2, "usec" => 0));
// should block to wait server response
$i = 0;
// $i < 10 is just for safety, not sure if its needed
// TODO: check if timed out recv returns FALSE or not
while ($i < 10) {
// 1800B should be enough? dont know MTU limits
$bytes_received = socket_recv($socket, $tbuf, 1800, 0);
if ($bytes_received === FALSE) die('An error occured while receiving from the socket');
$totalBytes += $bytes_received;
$buf .= $tbuf;
$i++;
//This is the ending sequence
if (strpos($tbuf,"\\EOT\x00\x00\x00") !== FALSE) {
break;
}
}
// close socket
socket_close($socket);
$split = explode("\\",$buf);
// echo masterlist
$i = count($split) - 1;
echo "Found $i servers. ";
while($i >= 0) {
$hexip = substr($split[$i],0,4);
$ip = hex2tostr(bin2hex($hexip));
$hexport = substr($split[$i],4,5);
$port = hex4tostr(bin2hex($hexport));

if ($ip != ''){
echo "$ip:$port ";
}
$first = 0;
$i--;
}
?>

Dont know what you mean by gametracker..

But plz this script shouldnt be publicly accessible, as it is accessing other's servers

wurst · **Posted:** 09.06.12

Hey, thx very much, thats better reading.

i dont wanna let people use it from here, id like to have page where people can browse urban terror servers.
features like real-time refreshing of a specified server status would be ok in my eyes.
important to collect timestamps and make it hammer/dos safe.

with gametracker i mean http://www.gametracker.com as the holy grail of online gameserver monitoring/tracking^^

oki what i did...
I made SQL backend, having tables for
- server (master) list
- extended server info (to be updated)
- amount of players graphing (to be continued)

now im having to query ~800 servers in narrow cycles.
script runs ~3 seconds, table fills up.
starting a vas amount of tasks means using multicore:

Attachment:

server_idle.png [ 2.17 KiB | Viewed 10893 times ]

Attachment:

server_query_800_gameservers_atm.png [ 2.5 KiB | Viewed 10893 times ]

1358 - 1061 = 297 MB
--> thats ok, anyways ~300KB Ram per process is something that could be better.

ah, heres the script that queries each gameserver:

Code:

<?php

// Turn lamish PHP E_NOTICE off!
error_reporting(E_ALL & ~E_NOTICE);

include './conf.php';

//SQL
$db = mysql_connect ($dbhost, $dbusername, $dbuserpass) or die(mysql_error());
mysql_select_db($dbname, $db) or die("Cannot select $dbname");

$vaiables = $_SERVER['argv'];
//print_r($vaiables);
$ip = $vaiables['1'];
$port = $vaiables['2'];

if ($ip == '' || $port == ''){ echo "usage: server_properties.php <server_ip> <server_port>\n"; die; }

$con = fsockopen("udp://$ip", $port, $errno, $errstr, 2);
fwrite($con, "\xFF\xFF\xFF\xFFgetstatus\x00");
stream_set_timeout($con, 2);

$s = "";

if ($con) {
$read = fread($con, 10000);
$s = substr($read, "4");
}
fclose($con);

// echo $s.' ';

$s_data = explode("\\", $s);

$response = $s_data[0];
//print_r($s_data);

if ($response == "statusResponse\n"){

$g_matchmode = explode("g_matchmode\\",$s);
$g_matchmode = substr($g_matchmode[1],"0","30");
$g_matchmode = explode("\\",$g_matchmode);
//echo "$g_matchmode[0]\n";

$g_gametype = explode("g_gametype\\",$s);
$g_gametype = substr($g_gametype[1],"0","30");
$g_gametype = explode("\\",$g_gametype);
//echo "$g_gametype[0]\n";

$sv_maxclients = explode("sv_maxclients\\",$s);
$sv_maxclients = substr($sv_maxclients[1],"0","30");
$sv_maxclients = explode("\\",$sv_maxclients);
//echo "$sv_maxclients[0]\n";

$sv_privateClients = explode("sv_privateClients\\",$s);
$sv_privateClients = substr($sv_privateClients[1],"0","30");
$sv_privateClients = explode("\\",$sv_privateClients);
//echo "$sv_privateClients[0]\n";

$g_matchmode = explode("g_matchmode\\",$s);
$g_matchmode = substr($g_matchmode[1],"0","30");
$g_matchmode = explode("\\",$g_matchmode);
//echo "$g_matchmode[0]\n";

$g_warmup = explode("g_warmup\\",$s);
$g_warmup = substr($g_warmup[1],"0","30");
$g_warmup = explode("\\",$g_warmup);
//echo "$g_warmup[0]\n";

$sv_hostname = explode("sv_hostname\\",$s);
$sv_hostname = substr($sv_hostname[1],"0","30");
$sv_hostname = explode("\\",$sv_hostname);
//echo "$sv_hostname[0]\n";

$fraglimit = explode("fraglimit\\",$s);
$fraglimit = substr($fraglimit[1],"0","30");
$fraglimit = explode("\\",$fraglimit);
//echo "$fraglimit[0]\n";

$timelimit = explode("timelimit\\",$s);
$timelimit = substr($timelimit[1],"0","30");
$timelimit = explode("\\",$timelimit);
//echo "$timelimit[0]\n";

$g_allowvote = explode("g_allowvote\\",$s);
$g_allowvote = substr($g_allowvote[1],"0","30");
$g_allowvote = explode("\\",$g_allowvote);
//echo "$g_allowvote[0]\n";

$sv_dlURL = explode("sv_dlURL\\",$s);
$sv_dlURL = substr($sv_dlURL[1],"0","30");
$sv_dlURL = explode("\\",$sv_dlURL);
//echo "$sv_dlURL[0]\n";

$Admin = explode("Admin\\",$s);
$Admin = substr($Admin[1],"0","30");
$Admin = explode("\\",$Admin);
//echo "$Admin[0]\n";

$Email = explode("Email\\",$s);
$Email = substr($Email[1],"0","30");
$Email = explode("\\",$Email);
// echo "$Email[0]\n";

//$uid = dechex("$ip$port");

//Generate UID from srv address
$ip_split = explode(".",$ip);
$uid = str_pad(dechex($ip_split[0]), 2, "0", STR_PAD_LEFT).str_pad(dechex($ip_split[1]), 2, "0", STR_PAD_LEFT).str_pad(dechex($ip_split[2]), 2, "0", STR_PAD_LEFT).str_pad(dechex($ip_split[3]), 2, "0", STR_PAD_LEFT).str_pad(dechex($port), 4, "0", STR_PAD_LEFT);

/*
echo "$uid\n";
echo "$ip\n";
echo "$port\n";
echo "$g_matchmode[0]\n";
echo "$g_gametype[0]\n";
echo "$sv_maxclients[0]\n";
echo "$sv_privateClients[0]\n";
echo "$g_matchmode[0]\n";
echo "$g_warmup[0]\n";
echo "$sv_hostname[0]\n";
echo "$fraglimit[0]\n";
echo "$timelimit[0]\n";
echo "$g_allowvote[0]\n";
echo "$sv_dlURL[0]\n";
echo "$Admin[0]\n";
echo "$Email[0]\n";
*/

$query = "
INSERT INTO `server_properties`
( `uid`, `port`, `g_matchmode`, `g_gametype`, `sv_maxclients`, `sv_privateClients`, `g_warmup`, `sv_hostname`, `fraglimit`, `timelimit`, `g_allowvote`, `sv_dlURL`, `Admin`, `Email`)
VALUES ('$uid', '$port', '$g_matchmode[0]', '$g_gametype[0]', '$sv_maxclients[0]', '$sv_privateClients[0]', '$g_warmup[0]', '$sv_hostname[0]', '$fraglimit[0]', '$timelimit[0]', '$g_allowvote[0]', '$sv_dlURL[0]', '$Admin[0]', '$Email[0]')
ON DUPLICATE KEY UPDATE
time_lastonline = UNIX_TIMESTAMP(CURRENT_TIMESTAMP)
;";
//echo $query;
mysql_query($query, $db);

echo "$ip:$port online";

}
else {

echo "$ip:$port offline";

}

//print_r($s_data);

?>

...And this shoots it:

Code:

<?php

// Turn lamish PHP E_NOTICE off!
error_reporting(E_ALL & ~E_NOTICE);

include './conf.php';

//SQL
$db = mysql_connect ($dbhost, $dbusername, $dbuserpass) or die(mysql_error());
mysql_select_db($dbname, $db) or die("Cannot select $dbname");

$query ="SELECT ip, port FROM server_list";

$result = mysql_query($query, $db);

while($row=mysql_fetch_array($result)){

//echo $row['ip']."\n";

$ip = $row['ip'];
$port = $row['port'];

shell_exec("php ./server_properties.php $ip $port > /dev/null 2>/dev/null &");

}

?>

btw.
thx blinky again for his great explode trix.
i know it eats ram but it accours to be the only way to have variable and value assigned fail safe.

#################EDIT#################

Every new Cycle on that 800 Servers brings me 800 new rows in the graphing table (where amount of players and current map are stored)
it makes ~50 KB in Database.
Doing it every minute would mean store 72 MB per Day / 2,1 GB per Month.
After, cummulating to 1 Value per hour reduces the amount of Data to additional 35 MB/ Month.
Thats Values where 1 can live with i would say...

#################EDIT#################
some output for registered users only...

masterlist.php

bky · **Posted:** 09.06.12

rly nice work.

maybe its just on the masterlist.php, there's something wrong with the actual map and players on srv:

Quote:

<<<Wurst test serwer 83.141.4.234:22222 50 9 ut4_dust2_v2

sidebar says ut4_suburbs and 22 players online

maybe ASC instead of DESC ordering for the result from the ..graph table?

SvaRoX · **Joined:** 11.29.08 **Posts:** 1972

Nice work !
Now I really need to install 4.2

playing with the mast0rlist

Who is online