<![CDATA[Rythie's Blog]]> 2019-11-23T12:56:00+00:00 https://rythie.com/blog// Octopress <![CDATA[Two Years with the Pebble watch]]> 2015-06-10T22:45:00-04:00 https://rythie.com/blog//blog/2015/06/10/two-years-with-the-pebble-watch

Smartwatches are still considered an emerging technology, yet if you backed the Pebble on kickstarter - like I did, you’ve been using this technology for two years already. The big debate around smartwatches, which like any new technology is, are they actually useful?

Features

On my watch I mostly use about 3 features:

1. Notifications

Notifications are the killer feature of smartwatch in my view. The pebble will show any notification that appears on my iPhone’s lock screen. At first this was quite noisy, so I prevented many apps (mostly games) from sending me notifications that weren’t useful. Now, I only enable notifications that might need a quick response for example text messages, Twitter @replies/DMs, calendar reminders etc. and nothing that doesn’t require a immediate response: emails, picture like/favourites. Quickly looking at these on my watch feels more polite than getting my phone out, though that might change over time.

2. Music control

The main other feature I use is music control, pause/next track and the name of the current track.

3. As a watch

The pebble works pretty well as a watch actually, there’s no need to wake or shake it, the time and date are shown all the time and some of watch faces are quite nice. The alarms feature is useful, just like on regular watch though when you stop and alarm it defaults to repeating the next day, which is annoying and the opposite of what iOS does.

Apps

I use a third party watch face, though I’ve yet to find an third-party app that I find genuinely useful.

Battery life

The battery lasts about 7 days, this is good enough that I only have one charger cable and I don’t normally take it with me anywhere. I charge it when I get the 20% battery life warning, though often I’m not near the charger when it happens, though that’s not a problem because it will still last all day on 20%.

General

In general there is a limit of 8 watch faces/apps that can be loaded at any one time on the watch, in practice this hasn’t been much an issue yet, I always find something I don’t use any more to make space. Support has been pretty good, when I first got the watch the charge cable was broken, but pebble sent a new one without much problem. Both the watch software and iPhone app are in very active development which is great for a product of this type.

What Next?

Not wanting to get behind, I’ve been looking at what to get next. I looked closely at the Apple watch, though I’m put off by the battery life, cost and the likely update within a year. It’s hard to see what value the Apple watch has above the pebble, though it might change as more apps emerge. So, I’ve backed the Pebble Time Steel, mostly due to good experience with this watch.

]]> <![CDATA[Why all websites should use HTTPS]]> 2014-03-05T08:20:00+00:00 https://rythie.com/blog//blog/2014/03/05/should-all-sites-use-https A recent survey revealed only 45% of the top 1 million sites have HTTPS enabled. The number is even lower if you consider that most of those sites are not available using HTTPS for all the content, but just a small bit, like the login page.

So what types of sites should use HTTPS all the time?

Banking sites

Banks, as you might expect, use HTTPS for their online banking. However, many users will click through from the information part of the site, which typically won’t use HTTPS. This creates a problem, a man in the middle could change that link to point at another site, and really would you notice the domain name was slightly different? Most of the big UK banks are getting there now, however, Lloyds still doesn’t allow HTTPS browsing of the main content - meaning you can’t really trust the online banking link unless you are very careful. I think the whole information site from a bank should be using HTTPS if only to provide a trustworthy link to the online banking.

Search engines

A few years ago it was common for search engines to not offer any kind of HTTPS access. At first this seems sensible, the results you get back are public and if you aren’t logged in, what’s the issue? The problem is what you search for, I think for most people, what they search for is private, people typically are giving away quite personal information to search engines, for kids this might be questions common in people growing up, in adults this might be questions about health that we don’t want to disclose to the world. What if we are using a public WiFi connection? others using that connection could find out what we are searching for, and embarrass or even extort us using the information we search for. Thankfully Google is now using HTTPS. Bing is now available as HTTPS, though unless you remember to change the URL, you won’t be using it. DuckDuckGo uses HTTPS by default and redirects if you try to use HTTP.

News sites

The first two examples were more obvious examples where HTTPS is needed, what about a news site? we don’t login, the information is public do we HTTPS here? The BBC is a big brand for news worldwide, and people should know if it is being censored. Without HTTPS there is no way for citizens in China, North Korea, Egypt and other places, to know that negative stories aren’t being carefully removed or altered by censors. Unfortunately trying to browse www.bbc.co.uk/news using HTTPS, just redirects you back to the HTTP site. Trying to do the same with the Guardian tells you the certificate is invalid - and if you look at the certificate (using firefox for example) you’ll find a big list of Fastly’s customers (the CDN the guardian seems to use).

Download sites

What about a site that offers downloads of free software? essentially again the information is public, but can you trust running software on your computer that could have been changed by a man the middle? For example if you were the NSA and you wanted to spy on someone, wouldn’t an easy option be wait insert your spyware into any executable they download? Of course there is code signing and checksums, though somehow I doubt most non-technical users check the checksums and most software is not signed anyway. GitHub gets this right, HTTPS all the time. Unfortunately sourceforge & download.com get this wrong. The PHP, Python and Ruby sites all fail to give you a HTTPS link to actually download the software.

Social Networks / Web-Apps

There are whole slew of sites that let you create some data and share or not share it with people. Often this data is fairly private outside our friends, so this should be protected from rogue wireless points, governments and the like. Facebook and Twitter now do this, Linkedin redirects back to HTTP straight after you login, as does RunKeeper.

So basically I think all sites should use HTTPS. In the web world we have various best practices, for example don’t use tables for layout, don’t write sites that only work in IE, or require flash. Making a site HTTPS should be one of these. I would be shocked if I saw sys admin using telnet in the 21st century and this should be true of making a non-HTTPS website.

]]> <![CDATA[Speeding up web API access]]> 2013-07-18T03:23:00+00:00 https://rythie.com/blog//blog/2013/07/18/speeding-up-web-api-access Web based APIs, such as the Twitter and Facebook APIs, are now commonly used in web applications. However, accessing these from a web application can quite easily slow it down, making it unresponsive to users.

This post is an overview of the major ways to make your application faster. This post is about application responsiveness, it’s not really about scalability (though nothing here should really prevent you from scaling). I’m not going to go into the technical details of these here, as there are plenty of resources on web.

Understanding the problem

HTTP based APIs typically take least 100ms per request due to network latencies. Multiple requests often mean you need several hundreds of milliseconds. Slow servers can balloon this to even longer, 10 seconds or more. Measuring the problems in your application, either using a profiling tool or with simple timings, is a good idea. Without measurements you risk wasting a lot of time optimising something that’s not a problem, whilst leaving the real problems untouched. Notably profiling can tell you if your biggest problem is really your API requests or something else, like your own database layer.

Cache it

Caching is the most obvious solution to the problem, i.e. store the result in a file somewhere (or use memcache) and get it again when someone else requests it. This can work well, if everyone viewing the site needs the same things from the API (i.e. it’s not personalised) and if that’s the case you can go one step further and fetch this data with a cron job. However it’s often not that easy! The big problems are that it’s not very good for personalised data and you are often presenting stale data to the user.

Fetch them at the same time

It’s quite likely that you will be requesting more than one endpoint from the API in a single page. The problem with this is, for each extra request you make the page will get slower. In most web programming languages nothing else can happen whilst these requests are in progress. The solution is to request these in parallel using cURL multi, this can be quite messy so I’d suggest using a library such as Rolling Curl [PHP]. Using cURL multi should mean that your requests only take as long as the longest one.

Be selective

Even though you can request multi endpoints in one go, it’s still best you don’t request stuff you don’t need, as unneeded requests can still slow down the page.

Don’t forget post-processing performance

It’s easy to think that your slow performance is completely caused by remote requests for API endpoints. However, try loading up a profiling tool (e.g. Xdebug if using PHP) to see where the performance bottlenecks really lie. For example if you are receiving a lot of results and running regular expressions on them, that could easily be the problem.

More than one way to do it

Up until this point I’ve assumed that you are requesting endpoints when you load the page, however there are a few other ways to do it.

AJAX

You can fire off an AJAX request which fetches the API endpoints, this can then be inserted into the page later on - typically this means the users see the main page load quickly, with the results from the API loading a little time after. The problem with this is that due to the use of Javascript to get the content, search engines won’t be indexing the content.

Cron

If suitable to your application you can use a cron to request end points at regular intervals and populate a cache, which is later read by the pages

Messages queues

Message queues can be useful if a simple AJAX or cron solution isn’t suitable, usually this is when a lot of requests are required. A message queue can be populated with the requests needed and something else can fetch these requests from the queue and process them.

Push methods

Lastly some APIs can actually push new data to you in real-time, such as Twitter streaming API. This means you never have to serve stale data again, however it’s not commonly available in APIs - also, it typically requires subscribing to certain subset of the data, which means it’s suitable to users who use your site regularly and requires caching of a lot of data that may never be accessed.

]]> <![CDATA[The ‘good enough’ threshold]]> 2012-07-06T00:00:00+00:00 https://rythie.com/blog//blog/2012/07/06/the-good-enough-threshold The term ‘good enough’ tends to be associated with lazy people justifying work that is sub-standard. However, when applied to products, it’s a powerful concept which decides when someone will switch from one type of product to a radically different one.

I should note that I have recently re-read the Innovators Dilemma, which inspired this post, you should read that if you haven’t already.

Let’s take the example of smartphones and their disruption of laptops and other computers. The main advantage of a smartphone is that is portable and mostly has data available everywhere. I had various phones before I eventually bought an iPhone 4, most of them were not good enough to read email let alone browse the web. My Nokia n96 (a top of the range phone at the time) for example would often take an entire 15 minute bus journey to catch up with email, by which time you had arrived and had another way to check email. At other times, I had tried to use the web browser on the n96 and it was so slow and rendering was so poor that it was easier and quicker to go home and check something using my computer than use the phone. This was a phone that was clearly not ‘good enough’ to challenge a laptop in either email or web.

My iPhone 4 however is the complete opposite. It’s actually quicker to check email using an iPhone than it is to turn on my computer and start an email program and same is true of many web browsing use cases. In fact, like most people, I find I use my iPhone (or iPad) at home in preference to my computers even when they are in easy reach.

So what is happening here? My computer has much faster CPU, more memory, a much bigger screen and vastly more storage space than the iPhone, but I’d rather use the phone? At a certain point I didn’t need more CPU or memory, in fact I already have too much of these and I’ve mostly got those things because they are so cheap, it’s crazy not to have them. In fact, I was just waiting for something in the phone form factor to be ‘good enough’ for me to switch much of my usage. In business meetings people are ‘getting by’ with iPhones and iPads because they are ‘good enough’ for 90% of their uses and means they don’t have to lug a big laptop around.

This disruption is why I think Intel is so interested in the ultrabook concept, because they have virtually no products in the phone/tablet market and they need to at least make notebooks viable again.

Disruptive products are all around, they usually have an compelling feature, but are poorer performers in other aspects, compared to the product they disrupt. People start using them when they get ‘good enough’ at that task. Some examples:

It’s often said that the best camera, is the one you have with you, the smartphone is that camera and now has ‘good enough’ image quality for most uses. The ‘always have it with you’ is a compelling advantage in addition to the ability to instantly upload to social networks.

The failure of SACD indicated that standard CDs were already ‘good enough’ and in fact, with the switch to MP3 players, people voted with their feet to say that MP3’s inferior quality sound was ‘good enough’ to replace CD, given the portability gained. Lossless high definition music has been viable for years, though there seems to very little demand for this, even now.

‘Good enough’ even happens in software, for example I am using Google docs to prepare this post, years ago it became good enough to replace OpenOffice for most of my uses and I use it in preference, due to the ability to pick up my documents wherever I am. Google docs didn’t need have to out-do traditional office packages for me to switch, it merely had to be ‘good enough’ combined with the key advantage of the different form factor (available everywhere - nothing to install).

Designers of disruptive, ‘good enough’ products have the advantage that they know what parts of the original product you value and which you don’t.

What products do you use now that are ‘good enough’ already?

]]> <![CDATA[Using a Hybrid of XMLReader and SimpleXML]]> 2011-02-27T00:00:00+00:00 https://rythie.com/blog//blog/2011/02/27/using-a-hybrid-of-xmlreader-and-simplexml
Parsing large XML files can be pain in PHP and most languages. Simon Hamp recently did a talk where he said he was using XMLReader for this purpose. Unfortunately, XMLReader suffers from the same problems that old XML parsers I used in 2001/2 had, in that they are not nice to code with. The problem with pull-parsers like XMLReader is method of programming is to get an event on each opening, closing tag, text section etc..

SimpleXML is much nicer to use but requires too much memory for large XML files. But, what if you could get the best feature of XMLReader (handing large files) with the best features of SimpleXML (readable code)?

So it turns out you can do this and I'm not the first, either. Many large XML files will simply be a large number of records, so if we assume for this example, each record is in a <item> tag, we can iterate through these and then export each record to SimpleXML for easy coding:

                $reader = new XMLReader();
                $reader->open($path_to_xml_file);
                while($reader->read())
                {
                        if($reader->nodeType == XMLReader::ELEMENT && $reader->name == 'item')
                        {
                                        $doc = new DOMDocument('1.0', 'UTF-8');
                                        $xml = simplexml_import_dom($doc->importNode($reader->expand(),true));
                                        echo $xml->title; //or whatever
                        }
                }

So what about performance? I created XML file 108Mb in size with 100,000 records and tried various methods, another with 500k records and one with 1500k records. Here is a graph of the number of records processed per second:

Chart_1

You can see the large slow down of SimpleXML which is caused by the large amount of swap the process was using. Measuring memory can be tricky, because PHP memory_get_peak_usage() function shows the same usage for each method, even though the PHP process baloons in size with SimpleXML, though it does mean your process doesn't hit the memory limit either. So instead I'm using the 'time' command line utility to measure the PHP processes' memory usage. XMLReader and my modified 'hybrid' method both use 33MB regardless of data size. SimpleXML uses 900MB just for the 108Mb/100k record file and it gets a lot worse as you can see:

Chart_2

A final word of warning, if you put the XML file in big string it will store the file in memory, as you might expect, which uses a fair amount of memory. As a result you should not use the xml() function of XMLReader to read content from a string but always use the open() function to read from the file or URL directly.

You can find the little benchmark I wrote for this on github.

 

]]> <![CDATA[What made you think iOS was open?]]> 2011-02-15T00:00:00+00:00 https://rythie.com/blog//blog/2011/02/15/what-made-you-think-ios-was-open People are up in arms about the news that Apple will take a 30% cut of any subscriptions. It's not really surprising since they take 30% of everything else, what's most surprising is that they want publishers to charge the same for in-app subscriptions as they do with those made outside. I guess the thinking is that there is still a cost of sale outside the apps through the advertising that is needed.

Apple has been on constant path of closing their products, every iPod, iPhone and iPad they have ever sold has had non-user replaceable battery and memory. In a market where most of the competitors provide such things, Apple is winning. After years of laptops with user-replaceable batteries, Apple gradually updated it's line to make all of them have non-replaceable batteries. Apple tells labels what they can and can't charge for music on iTunes, doesn't let iTunes connect to other manufacturer's players and doesn't let it's iOS devices connect to other music apps. I could go on, however with all these restrictions, people haven't voted with their feet and left for other products. Everything Apple does is becoming more closed and I don't know why people are surprised, did they think somehow that iOS was open?

When Apple launched the iPhone they made a few changes almost immediately, they dropped the 4GB model as people mostly bought the 8GB model and they dropped the price by $200. Clearly Apple made some mistakes there, people showed that they wanted a higher specification phone for a lower price than they anticipated. With the iPhone 3G they started accepting subsidies from carriers as is common with other phones. Andriod phones are starting to hit Apple's potential sales, I'm sure. I must have talked to half dozen people, if not more, whose primary reason to buy a Android phone is that's it's like the iPhone but cheaper. Phones are clearly price sensitive at the volumes Apple wants to sell at.

So what can Apple do about pricing? If they reduce the price of their devices they risk losing margin, Apple doesn't tend to do low margin products, so what to do? It's clear Apple is already working on the solution to this, take a cut of everything it can on it's platform, be that apps/games, music, movies, books and now magazine and newspapers.

Apple's model is much like the games console market, the consoles are cheap, even though the R&D and hardware costs are high. Console games are expensive however but after 30 years of that model consoles more popular than ever. If Apple, as rumoured, release a cheaper iPhone model I think it will be a clear dent in Android and the margin will most likely be made up by cost of apps, music, movies, newspapers etc. on the platform.

I don't think any of this is particularly good, however Apple is showing that they can do this even at this point when there a several competitors, which shows their dominance.

There is still however an open platform, one not controlled by anyone, one that works on many devices and doesn't require you to give anyone a cut, one that works on the iPhone too. It's called the web. Or failing that, Android is lot more open.

]]> <![CDATA[Clearing up the "Back to the Future" mis-information]]> 2010-07-05T00:00:00+00:00 https://rythie.com/blog//blog/2010/07/05/clearing-up-the-back-to-the-future-mis-inform There is meme going around that Doc Brown in the film "Back to the Future", was planning to travel forward to this day, before he got gunned down by the Libyans. It all started with this tweet by @totalfilm (official Twitter account of the Total Film magazine). 

First-tweet2
So what actually happened in the Film? Doc Brown is about to leave 1985 to go to the future, here's the quote:

"Marty: The future,? So where you going?

Doc: That's right, 25 years into the future. I've always dreamed on seeing the future, looking beyond my years, seeing the progress of mankind. I'll also be able to see who wins the next 25 World Series." (source)

Now, Back to the Future was first released in the USA on the 3rd of July 1985 and that's maybe where the confusion comes from. The current date as shown in the time machine is the 26 October 1985 so it's wrong. Total film did try to correct themselves, but, posting a photoshopped picture, was not the best way and attached tweet was unclear. The Guardian's blog also confirms that this incorrect.

It's now 7 hours after the original tweet and people are still passing on this information in vast numbers:

Tweet-search

Twitter we have a problem

Twitter is fast moving system and sometimes people make mistakes and I can understand that. Typically the way to resolve this is to send an updated message to all your followers admitting you made a mistake. The problem however, comes in when people start retweeting your incorrect tweet, you can't inform all those who received the incorrect tweet about the error. In fact it gets worse as people remove the original attribution and pass the tweet off as an original tweet of their own (as you can see it the screenshot above). Of course blogs don't have this problem, people simply update the post with new information as it comes to light. Facebook and blogs also have comments so the post can be shouted down for inaccuracies there, even before the post is updated.

So is this a problem? Well today it's some tweet about meaningless facts from a 25 year old film, but tomorrow it could be mis-information about someone's death for example, though it's not like that hasn't happened before.

Update:

There is also a nice video on Vimeo showing the actual dates entered into the time machine http://www.vimeo.com/13099195 [Update: Now taken down]

Update 2:

Please help stop the spread of mis-information by retweeting the following tweet 

 (links here) which should make it to the top of Twitter's search page if enough people do it.

Update 3:

There are numerous people now reposting the photoshoppedpicture that Total Film did, but of course the copies don't say it's fake, sigh.

Update 4:

Total Film have now written a proper apology about the hoax they inadvertely created.

Update 5:

I don't think Total Film even started this hoax, though they claim to have come it with all by themselves, it was all over twitter for a day previously. From looking at the comments in Total Film's post, and looking at the Twitter search history, I'm pretty sure that @gavmccaughey's tweet at 4:40AM GMT/UTC on the 4th of July, was the first tweet of his hoax and that itself was retweeted over 100 times. Thanks to Brown727's blog for the information.

Gav
Also we can see that is the first tweet to mention it (I've looked through several pages and no one was tweeting about it before):

Gav-search

 

]]> <![CDATA[iOS devices will and are disrupting the desktop space]]> 2010-07-01T00:00:00+00:00 https://rythie.com/blog//blog/2010/07/01/ios-devices-will-and-are-disrupting-the-deskt In 2007 when Apple unveiled the original iPhone, I was quite surprised they had used OSX as the basis for the operating system. It was notable that they were able use a fully featured operating system on a mobile phone, something that wasn't being done widely at the time. It marked the transition of mobile phones from simple devices to scaled down desktop computers. The iPhone OS as it was then known, seemed very basic given that it didn't support 3rd party applications, multi-tasking or copy and paste, however, it has quickly gained these features, this is one of the hallmarks of a disruptive technology:

"Disruptive innovation is a term used in business and technology literature to describe innovations that improve a product or service in ways that the market does not expect" - Wikipedia
The iPhone was disrupting to some low end uses of laptops by allowing web browsing and email on the move without a laptop (they weren't the first on email, but were on the providing the full web). The iPhone was better at these tasks than a laptop by virtue of being smaller, instant on, with GSM networking built in, GPS built in, no noise, no heat, no viruses and simpler to use.

The iPad has continued this disruption by replacing times when you would use a traditional laptop. In fact, in Technologizer's survey of iPad users, around 70% said they were frequently using their iPad instead of their computer. The iPad has the advantage of having a much longer battery life than a laptop, meaning you don't need to plug it in as often, or at all during normal use. Also, it benefits from a better form factor for web browsing which is better suited to the easy chairs and sofas where people would normally consume written content such books and magazines. The iPad of course has same benefit as the iPhone of being instant wake which typically laptops don't do - at least not well.

I believe that Apple is trying to solve it's own innovators dilemma. Apple's innovators dilemma is that computers are getting cheaper and the primary use for many home users is simply to browse the web and not to use many applications, meaning that any advantage Apple has in it's application base and ease of use, could be supplanted by a web browser only system. This will tend to lead to people buying cheaper, simpler devices that only do just that. Apple's problem that very few people will want to pay $1000+ for a Apple laptop, if the only application they ever use is the web browser. It's notable that none of Apple's iOS devices cost more than that $999 mark that it sells it's cheapest laptop for.

So that's the state of Apple's disruption against traditional desktop operating systems, so what's next?

Both iPhone and iPad already support a larger number of games than the Mac, but what if this could be scaled up? Apple could produce an iOS capable device that plugs into your TV, this would provide the same app store type system to directly compete with Nintendo in casual games as well as providing a media centre for watching online video, music, podcasts etc. Also, I wouldn't rule out Apple making a full TV themselves buying the display panel from someone else as they do with the iMac.

Apple could also produce a version of the iPad in the laptop form factor with a rotatable screen. For this type of device there would be the demand for proper multi-tasking support, Intel CPU and CD/DVD drive support, though of course this support already exists in the underlying operating system, Apple just needs to enable an interface in iOS to make it work.

In a similar vein Apple could introduce a version of the iMac form-factor running iOS. At some point iOS would have to stop depending on desktop PC for initial setup, syncing and upgrades as people would not have such as device in the future.

 

]]> <![CDATA[The problem with online advertising]]> 2010-03-08T00:00:00+00:00 https://rythie.com/blog//blog/2010/03/08/the-problem-with-online-advertising Ars Technica has posted about "Why Ad Blocking is devastating to the sites you love". However it is not us, the readers, that are to blame, it's the publishers and advertisers that have created adverts that are so bad, that we need sophisticated ways to work around them. I don't have the same problem with advertising on magazines and newspapers and on U.K. TV. I actually think some of the advertising is better than many of the programmes.

Why is web advertising so bad?

Lets start in early 90s when I started reading content on the web, back then there were very few ads, a few were animated gifs in banners but it was no major deal. In the early 00s ads got a lot more aggressive with popup adverts and even worse pop-under adverts. Pop-up and Pop-under adverts really destroyed the credibility of advertising for me, since they were actively making the web difficult to use and especially for non-technical people. Of course we solved this with Opera, Firefox and eventually with IE following suit so that every major browser supported blocking pop-ups.

Now, you'd think that advertisers would have learnt their lesson, don't make really adverts annoying or we will find some way to block them.

However they didn't, here are some of the behaviours that annoy me in online advertising today:

  • Animated content. This especially bad when it's in a column next to the content I'm reading, it's flashing in the corner of my eye and I can't read the content. So basically I can't read the content that was the whole purpose of my visit. Why not put this content at the top or bottom, so I can put it out of view while I'm reading?
  • Interstitial advertising. Now we don't have pop-ups, advertisers now think it's slide in a advert from the side over the content before I've read it or even worse cover the whole content with an advert before I can view it. This is again this is stopping me from using the site properly.
  • Ads that make a noise. In this day and age it's quite common for people to play music on their computers and therefore have speakers on or have forgotten to plug in their headphones. However in shared office it's disruptive and embarrassing to have adverts play sounds even when you click on them.
  • Poor quality products. For me anything where there is a prize, medical and dental products, weight loss etc.
  • Scammy ads that point to spyware to download. Really no site should have this and really it's the lesser sites that let the side down by continuing to do this.
  • Spinning up the CPU. I use Linux and the Flash support is not the greatest, in particular a Flash advert can often spin up to 100% CPU making they system noisy, slowing it down and wearing down the battery if it's a laptop.

If you are a content site and you don't do the above or you don't any more, then you probably think your safe, but you're not. People have long memories. What other sites do affects how people perceive advertising. People are tuning out the adverts everywhere they go, because they are so often irrelevant, so they don't even look at that part of the page.

Better Advertising?

I tried to think about the sites I love and how they handle advertising. The sites I love, are ones that have something so unique that is no replacement for their content. I would hate it if they closed down and would pay if asked to:

LWN.net

I read this because it has some good in-depth articles about kernel development and the only other way to get even close to the would be to read the very busy Linux Kernel Mailing List, which I don't have anywhere near the time to do. LWN actually has a paid option, without paying you get everything a week late, so I have the paid option.

Anandtech

I've followed Anandtech since it started in 1997 and it has consistently provided the most in depth coverage of hardware topics. In a way most of the articles are suggesting you buy some piece of hardware based on the reviews. However to keep the site neutral it advertises places that sell hardware, unfortunately most of the adverts are animated and that's annoying. They do have some context sensitive adverts that sometimes are fairly close to what is being talked about. I think they could innovate a bit here and have a list of the exact products in the review and where you could buy them. I think if the advertising revenue dropped they could do well buy partnering or being bought by a online computer parts retailer. Dpreview was bought by Amazon for example.

Techmeme

Techmeme is not a content site itself, but it a very good way to get an overview of what everyone is talking about. Techmeme have a brilliant model for advertising in that they simply have a section called "Techmeme Sponsor Posts" with some very good content from well known companies, which is often interesting enough to click on

Hackernews

Hackernews is a aggregation site, though it has a fair amount of content, I wouldn't want to be without it. It's not really the stories that come through, but the really good comments on every story that make the site. Hackernews doesn't have any advertising, though the whole thing is a big advertisement for the Ycombinator startup programme and Ycombinator funded startups. I think they must do pretty well out of this since they will surely see a much greater number of startups than their competitors.

Jason Calancis's this week in startups (TWiST) (Video podcast)

I think this content is simply unique, Jason basically brings in his friends to talk about startups and basically that's pretty much everyone in silicon valley and the surrounding area. The content is always very good, due to Jason and guests he has. Jason gets about 25,000 viewers to his show every week and brings in $4,000 every show, $1,000 each for the 4 advert slots he reads, this seems like a very good model to me. TWiST is also a good advert for OpenAngelForum which is Jason's own way to get startups funded, it's clear that promoting this is a genius move by Jason since a lot of the people doing startups watch his show and he knows all the best investors so he can really provide value doing that.

All this leaves me wondering, What's the future of web content? Is it paid content? More relevant, targeted advertising? Something else?
]]> <![CDATA[Don't buy a Nokia N96]]> 2009-09-17T00:00:00+00:00 https://rythie.com/blog//blog/2009/09/17/dont-buy-a-nokia-n96 The contract on my Sony Ericsson k810i phone expired at end of last year. I'd not had much luck with that phone, the joystick controller failed in the same way the it happened on the Sony Ericsson T610 I had on the one before last phone I had. Not being able to navigate the phone basically made it unusable towards the end. I figured, I'd had bad experiences with Sony Ericssons both times, I'd bought them, and I had three different Nokias and they had all worked pretty well. This time, I thought I'd go back to Nokia. The Nokia N95 got rave reviews, though for me, I was also looking for something to replace my aging 20GB iPod. The Nokia N96 was the successor to the N95 which added 16GB internal memory and the option to add another 16GB, which I did.

I also looked at the iPhone, though at the time it only supported 16GB, had no video support, no bluetooth stereo (which are all fixed now) and it's a pain to work with Linux since you need iTunes to get stuff on and off it. So I decided on the N96.

Nokia's phones used to be popular because they were easy to use and reliable, but they have really messed up big time with this one. Here some of the areas I've kept on hitting problems over the last 9 months with this phone:

Startup
When you need to turn on the phone it can take about a minute to start, also, slide to lock the screen doesn't work all this time. Normally this wouldn't be much of a problem but the phone does sometimes crash so you end up booting it more that you should need to.

Address Book
On an old Nokia phone I could type the first letter of a person's name and it would jump right to it, now they match the beginning of any word so, it will match their surname and other bits I've added to the name like "house", "work" "mob" etc. (maybe I shouldn't use these anymore) so searching for "h", "w" or "m" brings up a lot of unwanted results.

Text Messages
First off, sometimes when you receive a text the phone can take a while to wake up and show you the text at all. When your viewing the text you don't see the date/time of the text, on old non-smartphone Nokia phones you would simply scroll down, on the N96 you have to go to Options → (scroll right down to) Message Details and then scroll that down to see the time - which is crazy when the screen is so big.

Email client
The mail client leaves a lot to be desired, with the default settings it runs out of memory even when it is the only program running with an inbox of 4,000 messages. I found that reducing my mailbox down to about 1500 messages helped this a bit, but still most of the time wouldn't let you read a message. In the end, I found that limited the number of messages was the best answer which can be done with the following from Email:

Options → E-mail settings → Retrieval settings → Retrieval amount → From Inbox = 50 (default: All) Even with this setting, I find it can take several minutes to check for new message, when I haven't used it for a few weeks.

Web Browser
The web browser seems fine in general unless you have used iPhone or iPod Touch before. I have recently bought an iPod touch and it's really easy to do a web search, change the page URL, use tabs, bookmarks basically everything you need. Nokia's browser is very poor in comparison. When it has to load external stylesheets it seems to load that at the end so you start reading the un-styled content for a few seconds and there is a big jump while the page changes. There is no multiple tab support.

MP3 Player
The music player does work some of the time. Occasionally it gets confused on a song and it starts stuttering and this can be fixed by pausing and un-pausing it again. Also, sometimes the phone will just simply crash and reboot while your listening to a song. A minor issue is that on my Sony Ericsson k810i I could pause/play and change volume all with out unlocking the phone, which was useful, on the n96 I have to unlock it every-time to do this (they do provide a wired remote which I don't find useful, and my iPod had one which I never used).

I've talked to a few people with Nokia N96s, and they report the same type of problems. I can't see a good reason to choose the Nokia N96, so my recommendation is to get something else.

]]> <![CDATA[Federated short URLs]]> 2009-08-10T00:00:00+00:00 https://rythie.com/blog//blog/2009/08/10/federated-short-urls tr.im URL shortening service, the inevitable debate about shortening services has arisen. The key problem is that these URLs are lost if the service is shutdown and lots of links remain broken forever with no idea of what domain they were on let alone the specific page.

I must say, I'm not really convinced there is a need for URL shorteners outside Twitter, and they should provide a service of their own or avoid the need for it altogether. Nevertheless, there are lot of these services and here is what I think could be done to avoid a tr.im style problem again.

The solution would be a federated URL shortening system, such that all participants share the same shortening codes.
i.e.
http://bit.ly/Bxxxxx
would be the same as
http://tr.im/Bxxxxx

This is how I think it could work:

It's based on the same ideas of a hierarchical addressing used by IP addresses, postcodes, phone numbers etc.

We assume that these codes (i.e. the bit after the slash) are Base62 - i.e. each character could be a-z, A-Z or 0-9.
The shortening code would be split into two parts: a vendor part and a URL part.

i.e. bit.ly gets everything starting "B" leaving them 5 digits or 916 million addresses (625) to play with however they like.
trim gets everything starting "T" also leaving them 916 million addresses
Some small site could also get in the game with a smaller set initially, i.e. they get addresses starting "AAA" leaving them 3 digits or 238,328 (623) addresses to work though until they take off (which may never happen of course)

Everyone in the system would be responsible for approving short URLs onto the system for their subsection of the space. They would notify others in the system immediately and if a URL has already been shortened by another site, they would use the same code but serve it up as their own. I.e. tr.im finds that bit.ly has already shortened a URL as http://bit.ly/Bxxxxx so tr.im takes that code and serves it up as it's own http://tr.im/Bxxxxx and this way the user of the URL can get statistics and such. It's quite possible that if URLs are shortened on different sites within seconds of each other that two or more codes for the same URL will exist and that would probably less than 0.1% and could be accepted as an inherent inefficiency that would not be fixed.

As codes run out the prefix could be extended from 1 character to 2 or even 3, giving 62 times as many addresses in each case, giving multiple blocks to companies as necessary.

If a company wants to exit the game it would be suggested that it sells it's domain to an existing player. A DNS CNAME entry to the new company is all that would be really required to take it over, none of the code or database written by the failing company would need preserving and could even be sold separately.

Of course the failing company could either let the domain lapse or sell it to some company who didn't intend to carry on the service (probably redirecting to some scam or illegal site). In this case lets assume the shortener was http://fa.il/. In this case one thing that could be done is to have Twitter and other sites rewrite the old URLs and serve them up on another shortener. i.e. they rewrite http://fa.il/FFFFxxx as http://bit.ly/FFFFxxx and everyone's links keep on working.

So that's it. It does require a lot of companies working together which tends not to happen.

What do you think? Could this be done? Would anyone do it? ]]> <![CDATA[What Google's Chrome OS means]]> 2009-07-08T00:00:00+00:00 https://rythie.com/blog//blog/2009/07/08/what-googles-chrome-os-means Right now, if you work in a reasonably well run, large organization, I'd say your desktops are fairly well sorted out, backups are done automatically, you can roam with your login to any machine to get your desktop where you like, you can't mess much up or if you do the desktop can be reset within a day if not sooner. Data and/or desktop is available remotely. At least that's how I run the desktops I manage. Fundamentally I'd say in large organizations there is no real problem with desktop IT.

Now what about the other end of the spectrum? non-tech families, small non-tech businesses with no IT staff etc. How much backup is going on? can you roam from one machine to another in the house/organization? can machines be replaced quickly? what about remote access to documents/photos etc. - probably not.

It doesn't seem that either Microsoft or Apple have made massive inroads into this problem. Even with products like Apple's Time Capsule, your backups are in the same physical building as your Macs, so if your house burns down or your equipment is stolen, everything is lost.

I find that I already mostly have my data in the 'cloud' so to speak. I regularly switch between 3 systems my home desktop, my personal laptop and my work desktop. For any notes, lists, documents, spreadsheets I use Google docs & spreadsheets. I put photos on Facebook and Flickr. My bookmarks are in Xmarks and Delicious. My email is available via the web or IMAP. My programming code is generally stored on servers that I can SSH into from anywhere. In most cases I can get all the info I need, from anywhere in world, with a machine with a web browser and SSH terminal. There are a few exceptions, I can't easily access my music from anywhere and the photos I haven't published aren't available - though these are not unsolvable problems.

I think my experience has shown that you can store your data largely in the cloud. Google's Chrome OS, properly marketed, could popularize this type of computing. If Google provides a system on which you can't install programs, where all data is stored in the cloud, then people would start doing this. The benefits would be, the machine it's self is disposable, if the machine dies, you can buy another and no data would be lost - since it's in the cloud, being looked after by people who know what they are doing.

This is certainly not a new idea - though no-one of note has tried to push it. Microsoft and Apple were never likely to push it since it goes against the ecosystems they have created.

What do you think? is this Google's plan or are they are doing something else? What about offline? or does that matter anymore? Will people trust Google and others with their data? ]]> <![CDATA[Cutting features is not scaling]]> 2009-05-13T00:00:00+00:00 https://rythie.com/blog//blog/2009/05/13/cutting-features-is-not-scaling unauthenticated requests to 100/hour by IP. This stopped people from working around the per user limit which by then had also been raised to 100.

In April 2009, put a limit of follows per day, the most obvious reason for this is so that it puts a cap on "spam" following. I can see some other reasons though. On reason is to stop people following large numbers of people, I could see the logic that people won't just be limited to following 365,000 new people a year, but more likely they will just give up on the practice altogether.

Now twitter is limiting who some of your updates go to, so that replies don't go to most of your followers. This of course has caused a massive backlash typified by the #fixreplies trending topic on twitter. Twitter initially said it was doing this because it was confusing for users, even though it had been the default for months. Now twitter is saying it is a technical issue which makes more sense. I can see that when someone posts a reply now they will have vastly fewer places to update that which will cut the load. I suspect that under the hood twitter are redesigning database and this change is baked into it (Marshall Kirkpatrick from RWW has some thoughts on why that might be).

I can only think of one reason they might be changing it, and that's to allow threaded replies. Twitter now has in_reply_to_status_id field and clients are being updated to support it so you precisely what something was a reply to. You can't currently get the thread of replies easily. I wonder if twitter is moving toward storing these replies to an entry somewhere for easy retrieval so we can get the threaded replies.

Twitter is known to have various scaling issues from time to time and with these changes and other they seem to be limiting several features in order to put off hitting limit until another day. They don't seem have definite plan for scaling and cutting more and more features is not a solution. If you look at facebook for example which must have similar issues, they seem to have solved how to scale since they are at 200 million users and rarely have any downtime.

Twitter has about 30 employees, it seems they could do with some people that really know how to scale big sites i.e. people from facebook, myspace, friendster etc. but if you look at their jobs page, there doesn't seem to opening for a scaling expert. It's perhaps because they have the right people for already, but if that's the case, they should be scaling everything, not turning off features they can't make work. ]]> <![CDATA[Facebook pages vs. Twitter]]> 2009-03-05T00:00:00+00:00 https://rythie.com/blog//blog/2009/03/05/facebook-pages-vs-twitter Facebook's new "pages" features will compete with Twitter (and here)

It doesn't sound like what Facebook is doing will be much different than before, that said, often initially boring changes turn out to have interesting effects. Facebook is essentially improving it's pages system, so that you can follow various brands on Facebook. This is similar in way to going to a company's site and posting a comment on their forum, though without the sign up hassle, different interfaces and presumably I will be able to track replies better. Even with these simplifications, however, it will not be as easy as posting to Twitter. Facebook's approach is akin to looking up something using Yahoo's directory rather than searching for it on Google. On Facebook I will have to find somewhere appropriate to post my message (if it exists) and then post and probably not post if someone has made a similar comment.

User Interface
Twitter's system is, no system. It's the simplest system, since it a text box and button (like Google). So on Twitter I just post. I don't need to read what others said before I post. It's likely my followers haven't seen that yet and even if they have, multiple people saying the same thing strengthens the message anyway. Even though I haven't categorized my message through the power of search.twitter.com people will be able to find it, it may appear as part of a trending topic and someone from the brand can find if they so wish as can anyone else via keyword searches.

Immediacy
I can't imagine that Facebook's system is going to allow quick comments due to not having a limit on the message length and the categories system (as mentioned above). It seems for this reason Twitter is likely to remain the home of breaking news.

Discussion Types
There is another key difference in the way I expect people will use the sites other than the interface. On Twitter I am primary broadcasting messages to people on a variety of topics as they are to me. In Facebook's system, much like in a forum, I will be talking to a community of people that know the topic very well. This will make the process more daunting and less rewarding. For example, imagine going to a Britney Spears group to complain about her music when you know the group is full of Britney's Fans. You still want to make the comment but probably to a similarly like minded set of friends (such as your Twitter friends).

Freedom of speech
On Twitter I can say anything and only I can delete my comments. On Facebook, I expect as in common with most of their site, page owners will control the discussion allowing them to delete comments they don't particularly like. Brands will like this since they have some control, but users are likely to get frustrated by it.

In general Twitter and Facebook are so different I can't see the two competing on this. Twitter is going to have more regular, quicker, critical comments which are publicly searchable. Facebook's method is likely to attract a smaller proportion of it's users, who may in turn maybe more detailed due to Facebook's lack of message length limits and probably less critical. ]]> <![CDATA[Flickr shouldn't fear Facebook]]> 2008-10-20T00:00:00+00:00 https://rythie.com/blog//blog/2008/10/20/flickr-shouldnt-fear-facebook Stan Schroeder of Mashable posted a entry entitled Facebook Trumps Most Photo Sharing Sites With 10 Billion Photos. In it he says about Flickr that "ultimately people use Facebook for the same thing - sharing photos with their friends" and that "It’ll be interesting to see what photo sharing social networks will do to differentiate from Facebook".

It is possible that Stan still uses Flickr in this way, but I suspect many people already use large social networks (like Facebook) if they only want to share photos with their friends.

This is not a problem for Flickr, however. Flickr has established a community of people who want to share their photos with the world. Links to photos on Twitter or blogs often use Flickr for that purpose because twitter and blogs are by their nature public and Facebook is not. Through Flickr I have (re-)discovered photography, Flickr has a fascinating array of photos on it which can be easily found by going to the interestingness pages or one of numerous groups. Flickr also has a number of features which support it as a place to share photos with the world, not just your friends:

  • Tags
  • Public Groups
  • The API and various applications built on it (e.g. bighugelabs)
  • A community of photographers
  • Geo tagging
  • Search
(Facebook also has something called tags but that is for tagging people, which makes it useful for sharing with friends but is not the same as what Flickr uses tags for)

In short, Flickr is a way to share your photos with the world and Facebook is way to share photos with your friends. ]]> <![CDATA[Updated Digg stats]]> 2008-04-16T00:00:00+00:00 https://rythie.com/blog//blog/2008/04/16/updated-digg-stats ReadWriteWeb now featuring updated Digg graphs that I gave them, "The Decline and Fall of Tech on Digg". ]]> <![CDATA[phpMyAdmin 2.11.5 / PMASA-2008-1]]> 2008-03-01T00:00:00+00:00 https://rythie.com/blog//blog/2008/03/01/phpmyadmin-2115-pmasa-2008-1 security announcement. ]]> <![CDATA[More Digg stats]]> 2008-02-08T00:00:00+00:00 https://rythie.com/blog//blog/2008/02/08/more-digg-stats Tech stories percentage dropping on Digg", I thought I would do some more digg stats.
First up is the digg stories by number of popular stories rather than percentages:

I also looked at what Technology is made up of. It perhaps not surprising that a lot of the news is about Apple (dark blue) or Linux (light blue), with the generic "Industry News" making up the majority:
and here is the same data by percentage:
I wanted to get some of the data for submissions though it seems I can't get data from the API for that from before the 30th December, last year. ]]> <![CDATA[Tech stories percentage dropping on Digg]]> 2008-02-05T00:00:00+00:00 https://rythie.com/blog//blog/2008/02/05/tech-stories-percentage-dropping-on-digg Mashable today about the number of tech stories on Digg dropping, though the method they used was by looking at wayback machine results. However since Digg have all this information available in their API, we can get a much more accurate picture of what has happened. So here is a graph of the percentage of stories in each 'Container'. Each data point is for a full week of stories, so this should cover every story since the 1st of January 2006 until the 3rd of February 2008. (click on the image to view it full size)
Media_httpbp3bloggercomute2ye3dutgr6mwztkxy4iaaaaaaaaaboxzecouia50as400diggtechgraphpng_oiihvwavgckajoc

 ]]> <![CDATA[Loss of 25m child benefit records]]> 2007-11-21T00:00:00+00:00 https://rythie.com/blog//blog/2007/11/21/loss-of-25m-child-benefit-records 25 million child benefit records. It does seem that the data was password protected, but that presumably means something like a zip file or MS Excel password, then this is akin to losing a suit case with millions of pounds in it and then mentioning it had one of those tiny suit case padlocks on it.

It should be clear to any organization, that handles this kind of data, especially in this volume, that security is of paramount importance. Here some the security measures I think should have been in place and would have prevented this from happening:

  • No one should be able to download large portions of the data without special procedures for data security being enforced (say more than 1,000 records)
  • Any data that is transfered should be encrypted with strong cryptography. The key(s) to this encryption should be sent separately to the data and only after the data is acknowledged to arrived safely.
  • All data of this type should be sent by a secure transportation company such as Securicor.
  • If at all possible the data should never leave the secure site where it is normally resident
  • If anyone requests data the bare minimum that they require should be provided with no extra fields.
  • Anyone who goes anywhere near this kind of data should be given comprehensive security training.
Even if all but one of these measures were to fail, this breach would still not have happened.

The sad fact is that any security professional or group should have been able to tell them this (and whole lot more for other scenarios), but for some reason HM Revenue and Customs didn't take this type of advice or maybe didn't implement it properly.

Before this story surfaced I had already been very skeptical of the ID cards project. It would be very hard to keep this data safe. If the data were ever lost it would be a lot more serious than what has been lost in this case, both in the number of people and the amount of data about each person. I am now even more skeptical since it seems the government can't even handle quite basic data security concepts like the ones I have outlined. ]]>