Category: Programming

When I started writing this post I wasn’t sure should I put this into programming category or fun … I mean, validating if passed variable is integer aka whole number, how hard can it be? It appears not really easy in PHP 🙂

First off some background, I needed to validate an array with db key values (like array(1,2,6,7,…n)), so I thought of using simple function like array_filter with callback, something like:

array_filter($foo_array, 'is_int');

where is_int is callback, calling built in is_int function for each array value in order to filter out non int values.

The problem is (yes after 10 years dealing with PHP I am aware of that), PHP doesn’t treat int and string numbers the same, so string ’42’ wouldn’t be recognized as integer.

is_int('42'); //false
is_int(42); //true

To make things more “interesting” for programmers, if you have integer form data like age, year or whatever, they will be sent trough $_POST/$_GET/$_REQUEST arrays as ‘string numbers’ (so string ’42’ not int 42).

There is nice function to deal with such things and it’s name is is_numeric … but it only checks if string is actually number, so float values will be evaluated to true as well. ctype_digit on the other hand is opposite of is_int, it will only return true if test variable is string number, so ’42’ would evaluate to true, but 42 to false.

ctype_digit('42'); //true
ctype_digit(42);//false

And to make things even worse, PHP is silently converting really big integers to float (those bigger than PHP_INT_MAX const) so guess what would you get for number like is_int(23234234234235334234234)? Yep, false 🙂

$var = PHP_INT_MAX;
var_dump($var++); //true
var_dump($var); //false it's float now!

Yeah I know, you could cast var to int and do is_int … or cast var to string and do ctype_digit … and other dirty hacks… but what if someone smart from PHP team had decided to let say add 2nd argument to is_int check so you can check for type in some kind of ‘non strict’ mode, so string ’42’ is actually evaluated as integer? Something like this in C I guess:

function is_int(int var, int strict = 1)
{
   //if strict is false evaluate string integers like '42' to true!
}

All in all (at least to me), the easiest way to validate whether a variable is whole number (aka string or integer string) is with regular expression. Something like this:

/**
 * Test if number is integer, including string integers
 * @param mixed var
 * @return boolean
 */
function isWholeNumber($var)
{
	if(preg_match('/^\d+$/', $var))
	{
		return true;
	}
	else
	{
		return false;
	}
}

Someone from PHP dev team should really consider fixing this.

Today I had to deal with one huge table and cleanup all data where foreign key doesn’t have it’s primary key match in original table, just to remind myself how sub-queries in MySQL are terrible slower than joins.

I have some script which generates domains from typos, so I have one table with original domains (master_domains) and other one (result_domains) with generated typo domains. Basically something like this:

mysql> describe master_domains;
+--------+------------------+------+-----+---------+----------------+
| Field  | Type             | Null | Key | Default | Extra          |
+--------+------------------+------+-----+---------+----------------+
| id     | int(10) unsigned | NO   | PRI | NULL    | auto_increment | 
| domain | varchar(255)     | NO   | UNI | NULL    |                | 
+--------+------------------+------+-----+---------+----------------+
2 rows in set (0.07 sec)

mysql> describe result_domains;
+-----------+------------------+------+-----+---------+----------------+
| Field     | Type             | Null | Key | Default | Extra          |
+-----------+------------------+------+-----+---------+----------------+
| id        | int(10) unsigned | NO   | PRI | NULL    | auto_increment | 
| domain    | varchar(255)     | NO   | UNI | NULL    |                | 
| master_id | int(10) unsigned | YES  | MUL | NULL    |                | 
+-----------+------------------+------+-----+---------+----------------+
3 rows in set (0.01 sec)

Table result_domains has master_id which is foreign key reference to primary key (id) in master_domains table. Since I also have other scripts generating domains without typos (which store result_domains.master_id field as NULL), today I simple wanted to get rid of those masters without proper master_id reference in result table or in other words those master domains where result_domains.master_id is NOT NULL.

With sub-queries you could write query easily with something like this:

delete from master_domains where id not in 
(select master_id from result_domains_frontend)

It is good habit to always run select query before deleting big number of rows (just to make sure your query is written correctly) so I tried select query first:

select * from master_domains where id not in 
(select master_id from result_domains_frontend) limit 10

However, it took several minutes to run without any output so eventually I’ve decided to stop it. I know that sub-queries are much slower than joins, so decided to do try removal operation with left join.

Left joins are actually perfect weapon to find rows that exist in one (left) and doesn’t exist in other (right) table. They also have one big advantage over sub-queries – they are performing much faster, plus they are backward compatible with old prehistoric MySQL 5.x versions. However delete syntax is little bit tricky so after few trial and errors eventually I came out with this query:

delete master_domains.* from master_domains 
left join result_domains_frontend 
on master_domains.id=result_domains_frontend.master_id 
where result_domains_frontend.master_id is null ;

And voila after a while it came up with result:

mysql> delete master_domains.* from master_domains 
left join result_domains_frontend 
on master_domains.id=result_domains_frontend.master_id 
where result_domains_frontend.master_id is null ;
Query OK, 270558 rows affected (46.58 sec)
mysql> 

To move one table from one db to another, you can create new table (create table foo_new like foo) in db where you want to move table and then copy data with insert into/select query. However there is much easier way which is especially handy when you deal with big tables.

As you probably aready know, there is easy way to rename MySQL table just by issuing rename clause in alter statement:

ALTER TABLE foo RENAME TO new_foo;

You can also use RENAME TABLE syntax like this:

RENAME TABLE foo TO new_foo;

Now, when you need to move table from one db to another, all you have to do is to specify it’s current db and new db name as table prefix. For example if you want to move table foo from current db to new db you can issue queries like these:

ALTER TABLE currentdb.foo RENAME TO newdb.foo;

or

RENAME TABLE currentdb.foo TO newdb.foo;

Btw there is important difference between ALTER and RENAME statements in a way that with RENAME you can rename more than one tables at once. This comes handy if you want for example to swap names of two tables:

RENAME TABLE table1 TO temp, table2 TO table1, temp TO table2;

Recently, I had to change default behavior of storing session data into files and use MySQL DB instead. In practice, that means writing whole bunch of callback functions and setting callbacks with session_set_save_handler function. Since I use OOP, what really bothered me was the fact that (according to my PHP CHM Manual sitting on my desktop) for session_set_save_handler, all functions has to exist in global scope, since all callback arguments are strings?

bool session_set_save_handler ( string open, string close, string read, string write, string destroy, string gc )

Doing that in non OOP way with 6 functions on global scope is not something I really liked, so I googled for solution and found that you can easily assign an array like array(‘class_name’, ‘method’) for all callbacks in PHP. Cool stuff which allows you to create session handler class with bunch of static methods for those callbacks, but why the hell that is not documented in PHP Manual???

I went to online manual at least to see if someone submitted comment about this, and find out that session_set_save_handler definition there is completely different:

bool session_set_save_handler ( callback $open, callback $close, callback $read, callback $write, callback $destroy, callback $gc )

Obviously, since last time I browsed online manual, a lot of thing has changed, one among them is introducing “callback” type in those “pseudo types” used only for documentation purposes. And there, manual for callback says following:

callback

Some functions like call_user_func() or usort() accept user defined callback functions as a parameter. Callback functions can not only be simple functions but also object methods including static class methods.

A method of an instantiated object is passed as an array containing an object as the element with index 0 and a method name as the element with index 1.

Static class methods can also be passed without instantiating an object of that class by passing the class name instead of an object as the element with index 0.

which basically allows you to pass an array with class name and method as callback, and that method will be called.

Let me give you and example with sessions:

<?php

/**
 * Sessin_Handlers class
 * contains dummy methods needed for session stuff
 * Replace content with some real stuff like db conn etc.
 *
 */
class Session_Handlers
{
	function open($save_path, $session_name)
	{
		echo "Open Method Called<br>";
		return true;
	}

	function close()
	{
		echo "Close Method Called<br>";
		return true;
	}

	function read($id)
	{
		echo "Read Method Called<br>";
		return true;
	}

	function write($id, $sess_data)
	{
		echo "Write Method Called<br>";
		return true;

	}

	function destroy($id)
	{
		echo "Destroy Method Called<br>";
		return true;
	}

	function gc($maxlifetime)
	{
		echo "GC Method Called<br>";
		return true;
	}
}

//call all method from Session_Handlers statically
session_set_save_handler(array('Session_Handlers', 'open'), array('Session_Handlers', 'close'), array('Session_Handlers', 'read'), array('Session_Handlers', 'write'), array('Session_Handlers', 'destroy'), array('Session_Handlers', 'gc'));

session_start();

// proceed to use sessions normally
?>

As you see, we’ve created simple methods which only echo when they are called (in real life, you should either save session data into file or db). As you can see, we simple passed arrays to session_set_save_handler, which served us to connect class methods with session callbacks.

Although with release of PHP5 we finaly got some long awaited OOP features, sometimes I really miss overloading capability which exists in languages like Java. I am talking about something like this:

class Overloading_Test
{
  public void hello()
  {
    System.out.println("Hello Anonymous");
  }

  public void hello(String name)
  {
    System.out.println("Hello " + name);
  }


  public void hello(String firstName, String lastName)
  {
    System.out.println("Hello " + firstName + " " + lastName);
  }
}

This way you can call either hello with no arguments at all, or with one or two arguments, and proper method would always be called. Unfortunately, if you try something like this in PHP, it would give you fatal error, because basically, methods cannot be redeclared, since support for overloading is not part of core language like in Java.

However, there is still a way to achieve this Java like overloading functionality by using “magic” methods that are described in PHP Manual. Although it is not clear from manual how could you achieve exact functionality like in Java, I played a little bit with __call function, and get interesting workaround.

<?php

class Overloading_Test
{
  function __call($method_name, $arguments)
  {
    //list of supported methods
    //only 'hello' for this test
    $accepted_methods = array("hello");

    //in case of unexistant method we trigger fatal error
    if(!in_array($method_name, $accepted_methods))
    {
      trigger_error("Method <strong>$method_name</strong> doesn't exist", E_USER_ERROR);
    }

    //we inspect number of arguments
    if(count($arguments) == 0)
    {
      $this->hello1();
    }
    elseif(count($arguments) == 1)
    {
      $this->hello2($arguments[0]);
    }
    elseif(count($arguments) == 2)
    {
      $this->hello3($arguments[0], $arguments[1]);
    }
    else
    {
      return false;
    }
  }

  function hello1()
  {
    echo "Hello Anonymous<br>";
  }

  function hello2($name)
  {
    echo "Hello $name<br>";
  }

  function hello3($first_name, $last_name)
  {
    echo "Hello $first_name, $last_name<br>";
  }
}


$ot = new Overloading_Test();
$ot->hello();
$ot->hello("John");
$ot->hello("John", "Smith");
//this one will produce fatal error
//$ot->test();
?>

If you run this code, you will get something like:

Hello Anonymous
Hello John
Hello John, Smith

So, what is going on here? Whenever we call some undeclared method (which is the case with ‘hello’ method here), magic method __call is called, and two arguments (method name and arguments) are passed to it. For this simple test, we only support overloading of ‘hello’ method, so in case you try any other, we trigger fatal error.

What’s going on further is, we simple check number of argumens passed (by counting $arguments array), and call proper method. For the sake of clarity, I only used simple overloading based on number of arguments, but you could also check for argument type (ie string, integer etc.) and call proper method.

So, as you see, method overloading in PHP5 is not as elegant as in Java, but you can still make it. For more information about ‘magic’ fucntions (there are quite a few for member overloading as well), please visit PHP Manual.

If you ever dealt with backup of MySQL tables, you probably used mysqldump utility, which allows you to dump all data into some mytables.sql file, which you can backup somewhere, import to other MySQL server etc. I used this procedure for a thousands times so far, and generally it goes as follows:

export:

mysqldump -udinke -pmojpass -hhostname.of.server1 dbname table1 table2 tableN > dump_file.sql

mysql -udinke -pmojpass -hhostname.of.server2 dbname  < dump_file.sql

Sometimes there is a problem when you move data from new version of MySQL to old, and in that case you have to specify proper compability flag when running mysql dump (--compatible=name where name can be mysql323, mysql40, postgresql, oracle etc.).

Anyway, few days ago in order to move data from one MySQL to another, I had to dump some ... let say big mysql tables (about 10 tables, where each contained about 10 millions of records). After long lasted procedure (dump to file, gzip, scp to other server) I finally started import. However, after 3 hours (yes, three hours) instead of Linux prompt I got this:

[dinke@um-917 ~/public_html]$ mysql -udinke -p325ewfwt23rasf
keyword_discovery < es_miner_data.sql
ERROR 1582 (23000) at line 163833: Duplicate entry '1167548' for key
'PRIMARY'

WTF? I moved data from old server to new, which means import should run without compability problems. It could be indexes on old table were damaged, but instead to wait like 2 more hours in order to complete check & repair procedure, I've decided to create dump file, this time with ignore option, so all insert queries in dump file are "insert ignore", so in case of error like previous one, errors will be ignored. Not very smart, but those data are not really high sensitive, and I can afford to lose few records but can't afford to lose 10 hours for import!

So, dump, gzip, scp, import again ... which lasted long... loooooong .... so fucking long that after 3 hours after I started import I started to think about other solutions. And solution was dumb but effective. We simple moved all MySQL data files (*.MYI, *.MYD i *.frm) from one server to another, and then we run myisamchk in order to fix those tables because ... we didn't shutdown MySQL server during copy procedure which generally could cause some problems with data.

All in all, this procedure went very fast (the longest was actual copy from one host to another), and in less than half hour I got everything settled down.

At the end I came with conclusion that in case of really big tables, using tool like mysqldump is unaccepted as backup solution because it takes literally hours to complete. One of solution to that problem can be to copy MySQL data files like I did. That shouldn't be a problem, because tables are "platform safe" meaning, binary file created on one platform (ie *.MYI file on Linux) will work without problem when moved to other problem (ie. on Windows). The only one real problem is that if you copy data wihout MySQL shutdown, table files there are in some kind of "state of flux" which can be a problem, especialy if they are highly used in moment when you do copy. That's why you need to do myisamchk on it.

Yesterday I’ve spent considerable amount of time in order to find out more about the most interesting new PHP 5.2 feature – hook for upload progress meter. Except for this link I haven’t found anything else, no php source code example how to make one.

However, after I looked at internal php mailing list archive, I’ve found this thread, still no php data found, but Rasmus mentioned link with an example at http://progphp.com/progress.php. I immediately tried upload and it looked very cool. Then I looked at html source code and noticed one unusual thing there: APC_UPLOAD_PROGRESS hidden field inside of html source. I knew it must be important so I’ve googled for it, and insteresting enough first result was source code of Rasmus example above 🙂

So, I took complete source (I figured out later it is upload meter example made by Rasmus Lerdorf) and quickly tried to make it working under my fresh new installed PHP 5.2.0. Unfortunately, it didn’t work since it needed apc stuff installed. After I looked at apc documentation, I found that I need to grab it from list of pecl dll’s for windows php 5.2 version. Unfortunately, after I’ve downloaded it from here I’ve noticed apc dll is missing there ?!!

Again, I had to google for php_apc.dll and after a while found needed dll available at http://pecl4win.php.net/ext.php/php_apc.dll. In order to make it working, you have to save dll file under php/ext dir (i.e. c:\php\ext on windows) and put this to php.ini:

extension=php_apc.dll

Unfortunately, it still didn’t work, so I’ve looked at apc docs further. Finally on this page I’ve found apc have new “special feature” which is directly related to our new upload feature.

    apc.rfc1867             RFC1867 File Upload Progress hook handler is only available
                            if you compiled APC against PHP 5.2.0 or later.  When enabled
                            any file uploads which includes a field called
                            APC_UPLOAD_PROGRESS before the file field in an upload form
                            will cause APC to automatically create an upload_
                            user cache entry where  is the value of the
                            APC_UPLOAD_PROGRESS form entry.
                            (Default: 0)

After I figured out on my phpinfo page apc.rfc1867 setting is turned off, I’ve added

apc.rfc1867 = on 

in php.ini, and after restart was finally able to enjoy new fancy upload progress meter 🙂

Btw, upload also depend of json turned on as well, but it was already turned on so I didn’t have any more problems.

About the code Rasmus used in his example, I am tired to analyze it more now, but obviously it use Yahoo! User Interface Library to create progress bar and json/apc to control it from php during file upload.

I hope this will be helpful for someone. Enjoy 😉