playing with the mast0rlist

Everything technical and nerdy stuff here plz !

playing with the mast0rlist

Postby wurst » 09.03.12

He, im just sitting here and think about wether my server is on the masterlist.
After thinking some time more, i have this as a result:

Code: Select all
<?php


$version = $_GET['v'];

if     ($version == '42') {$master_udp_string = "\xFF\xFF\xFF\xFFgetservers 70 full empty\x00";}
elseif ($version == '41') {$master_udp_string = "\xFF\xFF\xFF\xFFgetservers 68 full empty\x00";}
else                      {echo "please use download.php?v=42 or download.php?v=41"; die;}


function hex2tostr($hex)
{
$str='';
for ($i=0; $i < strlen($hex)-1; $i+=2)
{
if ($i != 0) {$str .= '.';}
$str .= (hexdec($hex[$i].$hex[$i+1]));
}
return $str;
}


function hex4tostr($hex)
{
$str='';
for ($i=0; $i < strlen($hex)-1; $i+=4)
{
$str .= (hexdec($hex[$i].$hex[$i+1].$hex[$i+2].$hex[$i+3]));
}
return $str;
}


  $fp = fsockopen('udp://91.121.174.74', 27950, $errno, $error, 1);
  fwrite($fp, $master_udp_string);
  $read = fread($fp, 100000);
  $s = substr($read, "4");

  $split = explode("\\",$read);

// echo masterlist
$i = count($split);

while($i > 0) {
               
               $hexip   = substr("$split[$i]",0,4);
               $ip      = hex2tostr(bin2hex($hexip));
               $hexport = substr("$split[$i]",4,5);
               $port    = hex4tostr(bin2hex($hexport));
               
               if ($ip != ''){
               echo "$ip:$port <br>";
               }
               $first = 0;
               $i--;
              }

?>



Thats teh output:

dswp_tools/masterlist/download.php?v=41
dswp_tools/masterlist/download.php?v=42

so u might say now: cool!

however, theres a problem.
masterlist is sending me multiple UDP packets as an answer, the php just take 1 first packet from the response.
i know that fread() would limit the output, however checked that already...
Image
User avatar
wurst
Godlike
 
Posts: 4648
Joined: 07.15.08
Location: Behind U
-----tdm:  
nick: [dswp]GewitterOma
skill: 1122.83
kills: 25960
deaths: 19847
ratio: 1.30
-----bomb:  
nick: [dswp]GewitterOma
skill: 812.172
kills: 3885
deaths: 3541
ratio: 1.09

Re: playing with the mast0rlist

Postby wurst » 09.03.12

OKi here we go...

Code: Select all
<?php

$version = $_GET['v'];

if     ($version == '42') {$master_udp_string = "\xFF\xFF\xFF\xFFgetservers 70 full empty\x00";}
elseif ($version == '41') {$master_udp_string = "\xFF\xFF\xFF\xFFgetservers 68 full empty\x00";}
else                      {echo "please use download.php?v=42 or download.php?v=41"; die;}


function hex2tostr($hex)
{
$str='';
for ($i=0; $i < strlen($hex)-1; $i+=2)
{
if ($i != 0) {$str .= '.';}
$str .= (hexdec($hex[$i].$hex[$i+1]));
}
return $str;
}


function hex4tostr($hex)
{
$str='';
for ($i=0; $i < strlen($hex)-1; $i+=4)
{
$str .= (hexdec($hex[$i].$hex[$i+1].$hex[$i+2].$hex[$i+3]));
}
return $str;
}



  $fp = fsockopen('udp://91.121.174.74', 27950, $errno, $error, 1);
  @socket_set_timeout($fp, 1);
  fwrite($fp, $master_udp_string);
 
  $test = $byte = fread($fp, 1);
           while($test == chr(255)) {
           $socket_status = socket_get_status($fp);
           $byte_collect .= $test;
           while($socket_status['unread_bytes'] > 0) {
             //echo $socket_status['unread_bytes']; echo "<br>";
         
             $count++;
             $byte_collect .= $byte = fread($fp, 1);
             $socket_status = socket_get_status($fp);
           }
           socket_set_blocking($fp, 0);
           $test = fread($fp, 1);
           socket_set_blocking($fp, 1);
         }

// echo $byte_collect;
 

  $split = explode("\\",$byte_collect);

// echo masterlist
$i = count($split);


while($i > 0) {
               
               $hexip   = substr("$split[$i]",0,4);
               $ip      = hex2tostr(bin2hex($hexip));
               $hexport = substr("$split[$i]",4,5);
               $port    = hex4tostr(bin2hex($hexport));
               
               if ($ip != '' && $port != ''){
               echo "$ip:$port <br>";
               }
               $first = 0;
               $i--;
              }



?>


dswp_tools/masterlist/download.php?v=41
dswp_tools/masterlist/download.php?v=42

Result: 693 x Urt 4.11 vs. 201 Urt 4.2 servers 1 month after launch.

This helped me out:
http://www.php-resource.de/forum/php-de ... uffer.html
Image
User avatar
wurst
Godlike
 
Posts: 4648
Joined: 07.15.08
Location: Behind U
-----tdm:  
nick: [dswp]GewitterOma
skill: 1122.83
kills: 25960
deaths: 19847
ratio: 1.30
-----bomb:  
nick: [dswp]GewitterOma
skill: 812.172
kills: 3885
deaths: 3541
ratio: 1.09

Re: playing with the mast0rlist

Postby Samtron » 09.04.12

Can you also read out the amount of players? Would be interesting if there are also less 4.2 players than on 4.1.1 servers...
Image
User avatar
Samtron
Godlike
 
Posts: 1296
Joined: 01.08.09
Location: Berlin, Germany
-----tdm:  
nick: Samtron
skill: 584.92
kills: 316
deaths: 269
ratio: 1.17

Re: playing with the mast0rlist

Postby Unclefragger » 09.04.12

4.2 is officially released now? 8o
EDITH: just checked, 4.2 Beta is the "official" UrT now, but still beta ... =/
"Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live."
- Martin Golding
User avatar
Unclefragger
Godlike
 
Posts: 2006
Joined: 10.14.08
Location: rooftops
-----tdm:  
nick: [dswp]ucf
skill: 472.479
kills: 532
deaths: 332
ratio: 1.60

Re: playing with the mast0rlist

Postby bky » 09.04.12

BrainBug from php-resource.de wrote:sorry, nein, zu deinem Perl-Thema kann ich dir leider nicht weiterhelfen. Kenne mich mit Perl so gut wie gar nicht aus. :(


wursti hacking, nice :mrgreen:
Image
User avatar
bky
Godlike
 
Posts: 285
Joined: 08.31.08
Location: 1408
-----tdm:  
nick: TitaniusAngleSmith
skill: 1164.22
kills: 9429
deaths: 6605
ratio: 1.42
-----bomb:  
nick: TitaniusAngleSmith
skill: 886.644
kills: 963
deaths: 678
ratio: 1.42

Re: playing with the mast0rlist

Postby wurst » 09.04.12

why not hack urself today?

@no-of-player-per-server-question:

what the master server answers is just server addresses.
no name, slots, map etc.
however, if u look a bit to the right (or on top)
the scripts with that we monitor our servers speak "normal" rcon to the gameservers.
sure we can do so with any server, in theory.
practically this means working out a fancy concept that goes ok with 16GB Ram and 1-2 SSD.

as such a "gametrackish thingy" is missing for URT in separate versions, it could mean some load on the web interface.
specially cause i say no to advertisements, it should be a clean and fast HTML output.

@features/fancy concept:
- to query the masterlist is NOT resource intensive, its recieving 2-3-4 Packets and making dec from hex from bin.

-querying ~800-900 Servers from both lists same time means eating RAM.
If the script fires up inside a heavy apache2 1000 times: omg!
maybe it could make sense to have it in c, ill c.

-storing everything in mysql shouldnt be that problem, even in narrow monitoring cycles like 1 minute.
precision should be lowered over the time in datastore.
like storing 1 month every minute, 1 month - 1 yr ago every hour, etc.
output must be fucking fast, keeping database layout simple and flexible would be 1 important goal.

Additional Features that i miss on gametracker etc:
--> id love to have a green "this server is currently in masterlist and reachable"- button
--> non-gaming related tests like icmp ping/traceroute, reverse DNS.
--> maybe some attack tests like UDP rddos, q3infoboom et al, registration should be required, some extra work again.

What i wouldnt try:
collecting playerstats. sry, what the gametracker brings occour poor to me. when i compare with our beloved b3-truth.
I mean: RastaSkud has 171958 Kills. 171958? Really? i count 417166.
they counted 41% of his kills, 41,22052132724143 to be precise. precise? ah, right, there was something that says me: *u waste ur storage*.
players change their names, nobody will stop them.
i wouldnt be satisfied when i watch those numbers.
they would multiplicate the amount of stored data several times.
i dont wanna try what the gametracker does pretty well.
Image
User avatar
wurst
Godlike
 
Posts: 4648
Joined: 07.15.08
Location: Behind U
-----tdm:  
nick: [dswp]GewitterOma
skill: 1122.83
kills: 25960
deaths: 19847
ratio: 1.30
-----bomb:  
nick: [dswp]GewitterOma
skill: 812.172
kills: 3885
deaths: 3541
ratio: 1.09

Re: playing with the mast0rlist

Postby eXtr33m » 09.04.12

Got the sockets working... looks bit cleaner than rading by byte :D
Modified some snippet from php.net and wursti's code.
Code: Select all
<?php
if (!isset($_GET['v'])) die('Version not set.');
$version = intval($_GET['v']);
if     ($version == '42') {$master_udp_string = "\xFF\xFF\xFF\xFFgetservers 70 full empty\x00";}
elseif ($version == '41') {$master_udp_string = "\xFF\xFF\xFF\xFFgetservers 68 full empty\x00";}
else                      {echo "please use download.php?v=42 or download.php?v=41"; die;}
function hex2tostr($hex) {
   $str='';
   for ($i=0; $i < strlen($hex)-1; $i+=2) {
      if ($i != 0) {$str .= '.';}
      $str .= (hexdec(substr($hex,$i,2)));
   }
   return $str;
}
function hex4tostr($hex) {
   $str='';
   for ($i=0; $i < strlen($hex)-1; $i+=4) {
      $str .= (hexdec(substr($hex,$i,4)));
   }
   return $str;
}
if (!extension_loaded('sockets')) {
    die('The sockets extension is not loaded.');
}
// create socket
$socket = socket_create(AF_INET, SOCK_DGRAM, getprotobyname("udp"));
if (!$socket) die('Unable to create AF_INET socket');
// connect
$address = '91.121.174.74';
$address_port = 27950;
socket_connect($socket, $address, $address_port);
$msg = $master_udp_string;
$len = strlen($msg);
// at this point 'server' process must be running and bound to receive from serv.sock
$bytes_sent = socket_sendto($socket, $msg, $len, 0, $address, $address_port);
if ($bytes_sent == -1) die('An error occured while sending to the socket');
else if ($bytes_sent != $len) die($bytes_sent . ' bytes have been sent instead of the ' . $len . ' bytes expected');
$tbuf = '';
$buf = '';
$totalBytes = 0;
$bytes_received = 0;
// set timeout
socket_set_option($socket, SOL_SOCKET, SO_RCVTIMEO, array("sec" => 2, "usec" => 0));
// should block to wait server response
$i = 0;
// $i < 10 is just for safety, not sure if its needed
// TODO: check if timed out recv returns FALSE or not
while ($i < 10) {
   // 1800B should be enough? dont know MTU limits
   $bytes_received = socket_recv($socket, $tbuf, 1800, 0);
   if ($bytes_received === FALSE) die('An error occured while receiving from the socket');
   $totalBytes += $bytes_received;
   $buf .= $tbuf;
   $i++;
   //This is the ending sequence
   if (strpos($tbuf,"\\EOT\x00\x00\x00") !== FALSE) {
      break;
   }
}
// close socket
socket_close($socket);
$split = explode("\\",$buf);
// echo masterlist
$i = count($split) - 1;
echo "Found $i servers. <br />";
while($i >= 0) {
   $hexip   = substr($split[$i],0,4);
   $ip      = hex2tostr(bin2hex($hexip));
   $hexport = substr($split[$i],4,5);
   $port    = hex4tostr(bin2hex($hexport));

   if ($ip != ''){
      echo "$ip:$port <br />";
   }
   $first = 0;
   $i--;
}
?>


Dont know what you mean by gametracker..

But plz this script shouldnt be publicly accessible, as it is accessing other's servers :P
Last edited by eXtr33m on 09.06.12, edited 1 time in total.
User avatar
eXtr33m
Ingame Mod
 
Posts: 806
Joined: 12.19.09
Location: @Gauss:
-----tdm:  
nick: [dswp]Jan
skill: 330.877
kills: 220
deaths: 210
ratio: 1.04
-----bomb:  
nick: [dswp]Jan
skill: 664.203

Re: playing with the mast0rlist

Postby wurst » 09.06.12

Hey, thx very much, thats better reading.

i dont wanna let people use it from here, id like to have page where people can browse urban terror servers.
features like real-time refreshing of a specified server status would be ok in my eyes.
important to collect timestamps and make it hammer/dos safe.

with gametracker i mean http://www.gametracker.com as the holy grail of online gameserver monitoring/tracking^^

oki what i did...
I made SQL backend, having tables for
- server (master) list
- extended server info (to be updated)
- amount of players graphing (to be continued)

now im having to query ~800 servers in narrow cycles.
script runs ~3 seconds, table fills up.
starting a vas amount of tasks means using multicore:

server_idle.png
(2.17 KiB) Downloaded 10901 times

server_query_800_gameservers_atm.png
(2.5 KiB) Downloaded 10901 times


1358 - 1061 = 297 MB
--> thats ok, anyways ~300KB Ram per process is something that could be better.

ah, heres the script that queries each gameserver:

Code: Select all
<?php


// Turn lamish PHP E_NOTICE off!
error_reporting(E_ALL & ~E_NOTICE);

include './conf.php';

//SQL
$db = mysql_connect ($dbhost, $dbusername, $dbuserpass) or die(mysql_error());
mysql_select_db($dbname, $db) or die("Cannot select $dbname");

$vaiables = $_SERVER['argv'];
//print_r($vaiables);
$ip = $vaiables['1'];
$port = $vaiables['2'];

if ($ip == '' || $port == ''){ echo "usage: server_properties.php <server_ip> <server_port>\n"; die;  }


   $con = fsockopen("udp://$ip", $port, $errno, $errstr, 2);
   fwrite($con, "\xFF\xFF\xFF\xFFgetstatus\x00");
   stream_set_timeout($con, 2);

   $s = "";

   if ($con) {
      $read = fread($con, 10000);
      $s = substr($read, "4");
   }
   fclose($con);
   
//   echo $s.'<br><br>';
   
   $s_data = explode("\\", $s);

$response =  $s_data[0];
//print_r($s_data);

     if ($response  == "statusResponse\n"){

    $g_matchmode = explode("g_matchmode\\",$s);
    $g_matchmode = substr($g_matchmode[1],"0","30");
    $g_matchmode = explode("\\",$g_matchmode);
    //echo "$g_matchmode[0]\n";
     
    $g_gametype = explode("g_gametype\\",$s);
    $g_gametype = substr($g_gametype[1],"0","30");
    $g_gametype = explode("\\",$g_gametype);
    //echo "$g_gametype[0]\n";
     
    $sv_maxclients = explode("sv_maxclients\\",$s);
    $sv_maxclients = substr($sv_maxclients[1],"0","30");
    $sv_maxclients = explode("\\",$sv_maxclients);
    //echo "$sv_maxclients[0]\n";
     
    $sv_privateClients = explode("sv_privateClients\\",$s);
    $sv_privateClients = substr($sv_privateClients[1],"0","30");
    $sv_privateClients = explode("\\",$sv_privateClients);
    //echo "$sv_privateClients[0]\n";
     
    $g_matchmode = explode("g_matchmode\\",$s);
    $g_matchmode = substr($g_matchmode[1],"0","30");
    $g_matchmode = explode("\\",$g_matchmode);
    //echo "$g_matchmode[0]\n";
     
    $g_warmup = explode("g_warmup\\",$s);
    $g_warmup = substr($g_warmup[1],"0","30");
    $g_warmup = explode("\\",$g_warmup);
    //echo "$g_warmup[0]\n";
     
    $sv_hostname = explode("sv_hostname\\",$s);
    $sv_hostname = substr($sv_hostname[1],"0","30");
    $sv_hostname = explode("\\",$sv_hostname);
    //echo "$sv_hostname[0]\n";
     
    $fraglimit = explode("fraglimit\\",$s);
    $fraglimit = substr($fraglimit[1],"0","30");
    $fraglimit = explode("\\",$fraglimit);
    //echo "$fraglimit[0]\n";
     
    $timelimit = explode("timelimit\\",$s);
    $timelimit = substr($timelimit[1],"0","30");
    $timelimit = explode("\\",$timelimit);
    //echo "$timelimit[0]\n";
     
    $g_allowvote = explode("g_allowvote\\",$s);
    $g_allowvote = substr($g_allowvote[1],"0","30");
    $g_allowvote = explode("\\",$g_allowvote);
    //echo "$g_allowvote[0]\n";
     
    $sv_dlURL = explode("sv_dlURL\\",$s);
    $sv_dlURL = substr($sv_dlURL[1],"0","30");
    $sv_dlURL = explode("\\",$sv_dlURL);
    //echo "$sv_dlURL[0]\n";
     
    $Admin = explode("Admin\\",$s);
    $Admin = substr($Admin[1],"0","30");
    $Admin = explode("\\",$Admin);
    //echo "$Admin[0]\n";
     
    $Email = explode("Email\\",$s);
    $Email = substr($Email[1],"0","30");
    $Email = explode("\\",$Email);
    // echo "$Email[0]\n";
   
    //$uid = dechex("$ip$port");
   
    //Generate UID from srv address
    $ip_split = explode(".",$ip);
    $uid = str_pad(dechex($ip_split[0]), 2, "0", STR_PAD_LEFT).str_pad(dechex($ip_split[1]), 2, "0", STR_PAD_LEFT).str_pad(dechex($ip_split[2]), 2, "0", STR_PAD_LEFT).str_pad(dechex($ip_split[3]), 2, "0", STR_PAD_LEFT).str_pad(dechex($port), 4, "0", STR_PAD_LEFT);
   
    /*
    echo "$uid\n";
    echo "$ip\n";
    echo "$port\n";
    echo "$g_matchmode[0]\n";
    echo "$g_gametype[0]\n";
    echo "$sv_maxclients[0]\n";
    echo "$sv_privateClients[0]\n";
    echo "$g_matchmode[0]\n";
    echo "$g_warmup[0]\n";
    echo "$sv_hostname[0]\n";
    echo "$fraglimit[0]\n";
    echo "$timelimit[0]\n";
    echo "$g_allowvote[0]\n";
    echo "$sv_dlURL[0]\n";
    echo "$Admin[0]\n";
    echo "$Email[0]\n";
    */
   
                   $query = "
               INSERT INTO `server_properties`
                      ( `uid`,  `port`,     `g_matchmode`,     `g_gametype`,     `sv_maxclients`,     `sv_privateClients`,     `g_warmup`,     `sv_hostname`,     `fraglimit`,     `timelimit`,  `g_allowvote`,     `sv_dlURL`,     `Admin`,     `Email`)
               VALUES ('$uid', '$port', '$g_matchmode[0]', '$g_gametype[0]', '$sv_maxclients[0]', '$sv_privateClients[0]', '$g_warmup[0]', '$sv_hostname[0]', '$fraglimit[0]', '$timelimit[0]', '$g_allowvote[0]', '$sv_dlURL[0]', '$Admin[0]', '$Email[0]')
               ON DUPLICATE KEY UPDATE
               time_lastonline = UNIX_TIMESTAMP(CURRENT_TIMESTAMP)
               ;";
               //echo $query;
               mysql_query($query, $db);
   
    echo "$ip:$port online";
   
    }
    else {
   
    echo "$ip:$port offline";
   
    }
     

//print_r($s_data);

?>



...And this shoots it:
Code: Select all
<?php

// Turn lamish PHP E_NOTICE off!
error_reporting(E_ALL & ~E_NOTICE);

include './conf.php';

//SQL
$db = mysql_connect ($dbhost, $dbusername, $dbuserpass) or die(mysql_error());
mysql_select_db($dbname, $db) or die("Cannot select $dbname");


$query ="SELECT ip, port FROM server_list";

$result = mysql_query($query, $db);

while($row=mysql_fetch_array($result)){

//echo $row['ip']."\n";

$ip   = $row['ip'];
$port = $row['port'];

shell_exec("php ./server_properties.php $ip $port > /dev/null 2>/dev/null &");

}

?>


btw.
thx blinky again for his great explode trix.
i know it eats ram but it accours to be the only way to have variable and value assigned fail safe.


#################EDIT#################


Every new Cycle on that 800 Servers brings me 800 new rows in the graphing table (where amount of players and current map are stored)
it makes ~50 KB in Database.
Doing it every minute would mean store 72 MB per Day / 2,1 GB per Month.
After, cummulating to 1 Value per hour reduces the amount of Data to additional 35 MB/ Month.
Thats Values where 1 can live with i would say...

#################EDIT#################
some output for registered users only...

masterlist.php
Image
User avatar
wurst
Godlike
 
Posts: 4648
Joined: 07.15.08
Location: Behind U
-----tdm:  
nick: [dswp]GewitterOma
skill: 1122.83
kills: 25960
deaths: 19847
ratio: 1.30
-----bomb:  
nick: [dswp]GewitterOma
skill: 812.172
kills: 3885
deaths: 3541
ratio: 1.09

Re: playing with the mast0rlist

Postby bky » 09.06.12

rly nice work.

maybe its just on the masterlist.php, there's something wrong with the actual map and players on srv:
<<<Wurst test serwer 83.141.4.234:22222 50 9 ut4_dust2_v2

sidebar says ut4_suburbs and 22 players online :(

maybe ASC instead of DESC ordering for the result from the ..graph table?
Image
User avatar
bky
Godlike
 
Posts: 285
Joined: 08.31.08
Location: 1408
-----tdm:  
nick: TitaniusAngleSmith
skill: 1164.22
kills: 9429
deaths: 6605
ratio: 1.42
-----bomb:  
nick: TitaniusAngleSmith
skill: 886.644
kills: 963
deaths: 678
ratio: 1.42

Re: playing with the mast0rlist

Postby SvaRoX » 09.08.12

Nice work !
Now I really need to install 4.2 :)
ut4_he_tennis_v0.1
-------------------------
"We are talking about computers here, compared to those I can read women like a book ;P"
Unclefragger
User avatar
SvaRoX
Leader
 
Posts: 1972
Joined: 11.29.08

Next

Who is online

Users browsing this forum: No registered users and 0 guests

Misc