Archive for May, 2008

Graphing Social Patterns East 30% Off Promo Code

Friday, May 30th, 2008

GSP East Logo There’s a promo code up on the Facebook Group for GSP East right now from Dave McClure of 500hats fame (among other things). The code is gspe08fgd, but the breaking news is app developers who register now can get up to 50% off if they enter their app into the App Nite contest which is a 5 minute demo. Wish that 30% code was around when I registered!

If your on the fence about going to GSP East - now is the time to step up. Registration form is here I’ll see you there.

1990’s Dot Com: Where Are They Now?

Friday, May 30th, 2008

Industry Standard Logo Found this one on Slashdot this morning, and article from The Standard which looks at 10 stars of the 1990’s and where they (or their technology) are now. Personal favorites Pets.com and eToys which was a site I actually used and liked a whole lot better than anything out there today.

Quantcast Metrics

Friday, May 30th, 2008

Quantcast Logo I recently stumbled across Quantcast - a metrics tool. It provides more detailed information about visitor demographics than Alexa or Compete. It’s lacking otherwise in comparison tools - leaving it to the observer to draw their own conclusions. However, it’s a nice tool to add to an arsenal for reporting. The screenshot below is from Facebook, which ranks 16th overall on Quantcast but reaching 7th on Alexa.

Quantcast Screenshot Facebook.com

As with most of these advanced analytics tools, smaller sites which make up the majority of the internet are lacking information, so all numbers should be taken with a grain of salt. They do however provide a good indicator of trends and should be used as such.

Firefox Tools for Designers and Developers

Thursday, May 29th, 2008

Firefox Logo Mashable has a nice collection of 16 tools for designers and developers for plugging into Firefox. I’m personally a fan of Web Developer Toolbar and Y!Slow which are both great. The original Mashable article included the other two tools I find indispensable, ColorZilla and Firebug.

OpenSocial Could Learn Some Things From Facebook’s Platform

Thursday, May 29th, 2008

OpenSocial, a group effort to create a widget platform spear headed by Google, has a few glitches that I’d love to see fixed. They are largely comparison items from Facebook, who has recently announced that it will open source it’s own platform. Regardless of the motivation, there are a couple of items I’d like to see ported from Facebook to OpenSocial.

  1. Support for FBML like syntax: “Hold on a second! Standard HTML is the benefit of OpenSocial over Facebook” folks are probably thinking. While that’s true, without an effective way to capture and store user information for more than 24 hours (pesky terms of service documentation) developers are left querying for friends pictures etc over and over and over again. While this isn’t inherently a problem, if you wanted to display a 1,000 profile pictures for some unknown reason, you need to call the API a large number of times. Then generate the HTML and pass it to the client. This makes applications painfully slow. It’s great to store the numeric id (12345678) and be able to pass it back to the pre-client for parsing. The best examples are and . These really are very handy.
  2. Support for FQL like syntax: OpenSocial does a great job of providing methods for gathering most of the information you would want from the social graph, but it’s lacking in the ability to remix the data in new and interesting ways (easily). OpenSocial requires all of the heavy lifting to be done on the client (or the application backend if their API allows it). Most clients have a reasonable limit as to how big data structures can effectively be and have the application still function. Processing on the applications infrastructure negates another advantage of OpenSocial which is requiring very little in the way of hardware to operate.

There are also some learnings here for Facebook. I’d like to see a few OpenSocial conventions ported from OpenSocial to Facebook.

  1. OAuth Signature: To be fair, Facebook does provide signed requests, but it would be great if they’d use a standards based signature instead of their own homegrown version.
  2. External JavaScript Libraries: Facebook’s FBJS is powerful and provides most functionality that developers need. It’s even been open sourced so it can be used outside of the Facebook universe. However, developers who’ve been working with jQuery, ProtoType or any of the other numerous javascript libraries have to start at the beginning again. Additionally, they may be missing the functionality in the FBJS library that they need.

Both platforms still have a ways to go in terms of making developers life’s easier and users application experiences more robust. I think it’s great news that Facebook is opening their platform more. It’s really more symbolic than anything, because they still ultimately control what they do or do not implement on their platform. Bebo is the only other social network using Facebook’s model and it still requires some re-writing for developers because of syntax difference and lack of some features.

90 Twitter Related Tools and Applications

Tuesday, May 27th, 2008

Curious how big the extended twitter universe (twitterverse) is, I decided I’d do my own investigation and provide results I found as a list of tools. Basically, it’s BIG. Google returns ~69.3 million hits for “Twitter.” Below is what I found relevant in the cream. You’ll notice I skipped Mac/PC clients. There are lots of great clients that people use for platform specific updates (perhaps a future blog post worth).

Websites:

Spotting Trends (Techcrunch tweeted an even better one than these but I can’t for the life of me find it):

Tracking, Leaderboard, Ranking and Visualization:

Voice Tools:

iPhone Specific:

Web Browser Plugins (a very short list):

Social Networking Sites:

Mobile Applications:

Desktop Apps (Adobe AIR):

Search:

Photos & Images:

Mashups/Geolocation:

Blogging Tools (a very short list):

News & Politics:

Survey Tools:

Special thanks to franticindustries, Mashable and all of the assorted the commenters for the initial inspiration for this list. Also see Twitter Fan Wiki for a comprehensive list of apps. There are some big ones missing I’m sure, so YMMV with that list as with this one over time. All links/sites were active as of May 27, 2008.

Tiny Website Contest

Friday, May 23rd, 2008

Doing some reading this morning lead me to this site which has a collection of 10 bits of code that you can tweet. Reading this reminded me of the 5Kb website challenge. The premise was to create a functional website in 5Kb of code. That included all images, javascript, css, html… EVERYTHING! I recall there were a number of very complex graphic manipulation tools and even a shopping cart. Today I’m building out a new design for a client and my CSS include alone is already 3,197 bytes. This got me to thinking, we need a new 5Kb website challenge.

So here it is! What can you make in 5Kb?

Rules:

  • All HTML, CSS, JavaScript, Images and any initial httpRequest() content loaded at startup must be less than 5Kb
  • Content loaded via AJAX (if used) must be less than 5Kb per load request. In other words, it it loads a json object that references an image, the image + json must be less than 5Kb in size.
  • The site must “do” something.

Prizes

I’ll give the three (3) best entries a 512Mb Memorex Mini USB Thumb Drive. Entries must be submitted by June 16, 2008 to be eligible for a prize. Submit your site and description in the comments below.

Judging and all that good stuff is subject to participation. If we have enough entries, I’ll open it to the community - otherwise, I’ll be the judge. I’m off to get started on my (non-eligible) project.

JavaScript UUID Generator v. 0.2

Thursday, May 22nd, 2008

Nearly 2 years ago I crafted some JavaScript code to create a properly formatted UUID in JavaScript. Full compliance with specification for version 1 UUID’s still isn’t possible. However, in the interest of keeping the project updated, I’ve made some changes to the code to make it more reliable and more efficient.

  • It now use getTime() from the date object. This reduced the time spent calculating unnecessary values for the timestamp (idea came from James Hall’s canonical name function).
  • The randrange() function was replaced with one that more evenly distributes the random number values while additionally reducing complexity.

Get the download here.

I’d also like to give a shout to some of the folks who have mentioned it in forums, ported it to different libraries, linked to it from Wikipedia, blogs etc and have generally made my efforts worth while. Thank You!

View Code JAVASCRIPT
/*
 
uuid.js - Version 0.2
JavaScript Class to create a UUID like identifier
 
Copyright (C) 2006-2008, Erik Giberti (AF-Design), All rights reserved.
 
This program is free software; you can redistribute it and/or modify it under 
the terms of the GNU General Public License as published by the Free Software 
Foundation; either version 2 of the License, or (at your option) any later 
version.
 
This program is distributed in the hope that it will be useful, but WITHOUT ANY 
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A 
PARTICULAR PURPOSE. See the GNU General Public License for more details.
 
You should have received a copy of the GNU General Public License along with 
this program; if not, write to the Free Software Foundation, Inc., 59 Temple 
Place, Suite 330, Boston, MA 02111-1307 USA
 
The latest version of this file can be downloaded from
http://www.af-design.com/resources/javascript_uuid.php
 
HISTORY:
6/5/06 	- Initial Release
5/22/08 - Updated code to run faster, removed randrange(min,max) in favor of
          a simpler rand(max) function. Reduced overhead by using getTime() 
          method of date class (suggestion by James Hall).
 
KNOWN ISSUES:
- Still no way to get MAC address in JavaScript
- Research into other versions of UUID show promising possibilities 
  (more research needed)
- Documentation needs improvement
 
*/
 
// On creation of a UUID object, set it's initial value
function UUID(){
	this.id = this.createUUID();
}
 
// When asked what this Object is, lie and return it's value
UUID.prototype.valueOf = function(){ return this.id; }
UUID.prototype.toString = function(){ return this.id; }
 
//
// INSTANCE SPECIFIC METHODS
//
 
UUID.prototype.createUUID = function(){
	//
	// Loose interpretation of the specification DCE 1.1: Remote Procedure Call
	// described at http://www.opengroup.org/onlinepubs/009629399/apdxa.htm#tagtcjh_37
	// since JavaScript doesn't allow access to internal systems, the last 48 bits 
	// of the node section is made up using a series of random numbers (6 octets long).
	//  
	var dg = new Date(1582, 10, 15, 0, 0, 0, 0);
	var dc = new Date();
	var t = dc.getTime() - dg.getTime();
	var h = '-';
	var tl = UUID.getIntegerBits(t,0,31);
	var tm = UUID.getIntegerBits(t,32,47);
	var thv = UUID.getIntegerBits(t,48,59) + '1'; // version 1, security version is 2
	var csar = UUID.getIntegerBits(UUID.rand(4095),0,7);
	var csl = UUID.getIntegerBits(UUID.rand(4095),0,7);
 
	// since detection of anything about the machine/browser is far to buggy, 
	// include some more random numbers here
	// if NIC or an IP can be obtained reliably, that should be put in
	// here instead.
	var n = UUID.getIntegerBits(UUID.rand(8191),0,7) + 
			UUID.getIntegerBits(UUID.rand(8191),8,15) + 
			UUID.getIntegerBits(UUID.rand(8191),0,7) + 
			UUID.getIntegerBits(UUID.rand(8191),8,15) + 
			UUID.getIntegerBits(UUID.rand(8191),0,15); // this last number is two octets long
	return tl + h + tm + h + thv + h + csar + csl + h + n; 
}
 
 
//
// GENERAL METHODS (Not instance specific)
//
 
 
// Pull out only certain bits from a very large integer, used to get the time
// code information for the first part of a UUID. Will return zero's if there 
// aren't enough bits to shift where it needs to.
UUID.getIntegerBits = function(val,start,end){
	var base16 = UUID.returnBase(val,16);
	var quadArray = new Array();
	var quadString = '';
	var i = 0;
	for(i=0;i<base16.length;i++){
		quadArray.push(base16.substring(i,i+1));	
	}
	for(i=Math.floor(start/4);i<=Math.floor(end/4);i++){
		if(!quadArray[i] || quadArray[i] == '') quadString += '0';
		else quadString += quadArray[i];
	}
	return quadString;
}
 
// Numeric Base Conversion algorithm from irt.org
// In base 16: 0=0, 5=5, 10=A, 15=F
UUID.returnBase = function(number, base){
	//
	// Copyright 1996-2006 irt.org, All Rights Reserved.	
	//
	// Downloaded from: http://www.irt.org/script/146.htm	
	// modified to work in this class by Erik Giberti
	var convert = ['0','1','2','3','4','5','6','7','8','9','A','B','C','D','E','F','G','H','I','J','K','L','M','N','O','P','Q','R','S','T','U','V','W','X','Y','Z'];
    if (number < base) var output = convert[number];
    else {
        var MSD = '' + Math.floor(number / base);
        var LSD = number - MSD*base;
        if (MSD >= base) var output = this.returnBase(MSD,base) + convert[LSD];
        else var output = convert[MSD] + convert[LSD];
    }
    return output;
}
 
// pick a random number within a range of numbers
// int b rand(int a); where 0 <= b <= a
UUID.rand = function(max){
	return Math.floor(Math.random() * max);
}
 
// end of UUID class file

Getting Accurate Metrics in WordPress

Thursday, May 22nd, 2008

WordPress + Google Analytics Google Analytics does a great job keeping track of visitors, sessions, geographic locations and with the addition of the Benchmarking feature, even compares your traffic to other blogs of similar size and topic. However, there’s a problem. It counts you too! If your using a fancy plug-in to manage your Google Analytics account, you won’t need this. I, however, have been tweaking and tuning my template over time and have my it all stuffed right in header. Today I made a minor tweak to my WordPress template it to suppress my views when I’m logged in and updated to the new tracking code from the old Urchin based code. This snippet is from my ./wp-content/templates/header.php file.

<?php if($user_ID != 1){ ?>
<script type="text/javascript">
var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
</script>
<script type="text/javascript">
var pageTracker = _gat._getTracker("UA-123456-1");
pageTracker._initData();
pageTracker._trackPageview();
</script>
<?php } ?>

Be sure to add the id (or id’s) you want to suppress and to put the correct Google Analytics account information (the UA-123456-1 bit) in your code.
Happy tracking ;)

Not So Fast! MySQL Stored Procedures Are Not Panacea

Wednesday, May 21st, 2008

MySQL Logo MySQL 5.0 introduced stored procedures, but are you taking advantage of them? MySQL claims a 50% lift in performance just having an execution plan on hand, but what real world performance boost can you expect to see? I set out to discover just that and found the results unsatisfying.

Column Type Indexed
id int(11) unsigned Primary Key
content varchar(4096) No

I then conducted a test which would closely mimic a real world example to determine just that. I started with a table with a few million rows of variable length data in it. I then created a stored procedure that would as closely as possible match my hand coded SQL query. Then I created a simple PHP script to test it all out.

CREATE PROCEDURE sp_getdata (i INT(11))
BEGIN
	SELECT * FROM test_table WHERE id = t;
END;
$iterations = 100000;
$conn = mysqli_database($server, $user, $pass, $schema);
 
// find the extents of our random range and make sure the connection actually opens
$max = $conn->query("select max(id) as maxid from test_table");
$max = $max->fetch_assoc();
$max = $max['maxid'];
 
$sql = "SELECT * FROM test_table WHERE id = %d";
$procedure = "CALL sp_getmessage(%d)";
 
// run the stored procedure query
for($i=0;$i<$iterations; $i++){
     $query = sprintf($procedure, rand(1,$max));
     $conn->multi_query($query);
     $result = $conn->store_result();
}
print number_format(microtime(true)-$t2,2) . "s\n";
 
// run the straight sql query
print "Test SQL Query: ";
$t1 = microtime(true);
for($i=0; $i<$iterations; $i++){
     $query = sprintf($sql, rand(1,$max));
     $result = $conn->query($sql);
}
print number_format(microtime(true)-$t1,2) . "s\n";

I then ran the test 5 times to eliminate the so called warm-up period which MySQL would inevitably use to more accurately answer future questions. I then ran the test a number of times swapping the position of the two “tests” within the code so that neither received an unfair advantage. The results are below:

Run SQL Stored Procedure Order
1. 0.28s 0.39s Proc/SQL
2. 0.28s 0.37s Proc/SQL
3. 0.28s 0.41s Proc/SQL
4. 0.28s 0.38s Proc/SQL
5. 0.28s 0.39s Proc/SQL
6. 0.28s 0.80s SQL/Proc
7. 0.28s 0.82s SQL/Proc
8. 0.28s 0.81s SQL/Proc
9. 0.28s 0.82s SQL/Proc
10. 0.27s 0.80s SQL/Proc

I was taken by surprise by two obvious trends. First, the SQL statement was amazingly consistent regardless of when it was run. Second, not only was the stored procedure slower, it was significantly slower if a batch of plain old SQL statement executions were run first.

Next I decided to track the response time of each individual query that I ran to get a better look at the execution time costs for each. I expected to see a bell curve showing some queries running very fast and others running very slow with the majority falling in the middle somewhere. I had to increase the sensitivity of my timer millionths of a second to get a good picture of just how fast things were or were not running. Again I was surprised by the results. The basic query was still much faster in total elapsed time, but suddenly the performance of the stored procedure began showing itself. I ran the tests the same way as before and swapped the order in which they ran to eliminate preferential execution order. The following table gives results from one run of 100,000 so you can see the results first hand. The method was to insert the time value into an array and increment the value of that index by one each time a query executed in that time. So the first value you see is also the first query that gets sent to the server. This is important because for the stored procedure, you’ll notice it’s MUCH longer than all subsequent runs. Oddly, the query to get the maxium value had already been run so I’m not sure what causes the increase delay in runtime.

Stored Procedure Basic SQL Statement
    [0.00144] => 1
    [0.00005] => 2
    [0.00002] => 52
    [0.00001] => 99938
    [0.00003] => 6
    [0.00004] => 1
    [0.00063] => 1
    [0.00040] => 1
    [0.00047] => 16
    [0.00049] => 42172
    [0.00048] => 26
    [0.00299] => 21
    [0.00050] => 1735
    [0.00124] => 88
    [0.00074] => 517
    [0.00249] => 32
    [0.00224] => 38
    [0.00174] => 171
    [0.00149] => 138
    [0.00274] => 22
    [0.00099] => 139
    [0.00324] => 19
    [0.00399] => 12
    [0.00374] => 12
    [0.00349] => 12
    [0.00199] => 102
    [0.00054] => 1
    [0.00043] => 1
    [0.00075] => 35
    [0.00046] => 5
    [0.00250] => 2
    [0.00024] => 49970
    [0.00100] => 4
    [0.00025] => 4657
    [0.00125] => 6
    [0.00023] => 11
    [0.00175] => 2
    [0.00022] => 6
    [0.00449] => 4
    [0.00028] => 2
    [0.00052] => 1
    [0.00044] => 1
    [0.00200] => 1
    [0.00030] => 1
    [0.00041] => 1
    [0.00026] => 3
    [0.00027] => 4
    [0.00070] => 1
    [0.00499] => 1
    [0.00150] => 2
    [0.29807] => 1
    [0.00275] => 1
    [0.00424] => 2

Clearly the fluctuation in times we see in the first table are largely related to the first query we execute. After that initial procedure call, stored procedures run consistently fast, faster in fact than the plain old SQL query. But back to the “real world test”. How often do we create a single page that executes 100,000 queries at a time? More likely its 2-20 queries which are all very different than the others in the group. Given that variable, which is really the best solution? Ultimately it’s going to depend on your setup. This was a mostly idle app server communicating over a 100Mb connection to a moderately loaded DB server with ample RAM and disk performance.

Future exploration areas will be to research the most efficient way to execute stored procedures for maximum performance and deciding if a stored procedure is faster for more complex queries involving joins etc. If anyone has some good directions I can head in here, I’m all ears.

© 1998-2008 AF-Design, All rights reserved.