June 29th, 2008

“Quantity has a quality of its own”

I saw this quote on some swingers site. I like it a lot. It reminds me somewhat of Milan Kundera in The Unbearable Lightness of Being where he talks about there being two types of womaniser; first there is the man who is looking for the perfect woman but of course never finds her and secondly the man who just enjoys the infinite variety.

It is also something that is very true about marketing. Providing you have a basic proposition that people like then the key to successful promotion is just quantity; the more you do the more it works.

Fab Swingers has succeeded because of quantity. It’s not just quantity in marketing of course, but there is also a strong network effect because the more members the site has, the more valuable it is to the members. And the more likely they are to recommend it.

The only thing I’m feeling a bit negative about right now is that I’m mainly having to focus on just keeping the site going as the servers struggle to cope with the rocketing traffic. I’d be much happier if I could concentrate on marketing which is my skill.

However I’m sure that most social networking sites would be happy to have the problem of struggling to cope with traffic growth!

June 28th, 2008

Turning Lighttpd access log off for performance

I took a look at the Lighttpd access log, which is actually more than 1 GB a day. This is completely useless, it’s taking lots of disk space and it’s far too big for me to ever run an analysis program on. Not only that but it’s only going to get worse!

As I’ve studied the performance of the server generally it’s pretty clear that it is very IO bound (although things can go a bit slow sometimes, the processors are always mostly idle). So writing a 1 GB log each day isn’t going to help.

I do want some stats so I enabled the statistics module which tells me that lighttpd is averaging about 60 hits per second (although right now it’s 75 hits per second).

Since I made the change I think that performance has improved a bit but it’s not been massively significant.

I am also considering moving Lighttpd off to another server completely, although it uses almost no CPU I think that it is using up IO bandwidth. I really need another server anyway but I’ll wait until the July promotions start at The Planet.

June 22nd, 2008

MyISAM merge tables to improve performance

I use a login MyISAM table to record all logins to the site (IP address, user agent, username, etc) which is pretty handy for spotting scammers and people with multiple accounts.

Anyway, the problem is that this table gets very large, very quickly. My solution had simply been to run a delete every month to remove older entries. But this led to two problems:

  1. The table was locked for ages while the delete happened.
  2. The table was then left fragmented (which prevents concurrent inserts on my configuration)

My first thought was to use DELETE … LIMIT 100; in a loop so that the table wouldn’t be locked for too long. However that was still not an ideal solution as it was still using a lot of resource and fragmenting the table.

What I finally settled on was using a new login table for each month and then using a MyISAM merge table to join them all together. Then, I can simply drop the old tables as they go out of date and reconfigure the merge table.

I can now delete millions of rows instantly with no performance impact or fragmentation. Result!

June 17th, 2008

Various bits of MySQL performance improvement

It’s slightly embarrassing but I’ve actually slightly fallen for the old problem of going into optimizations without properly measuring. To be fair, part of the problem is that MySQL does not have the greatest reporting tools so it is not that as easy to spot the problems as it is with other databases.

Anyway, I identified that one of the problems was that some users just had too many messages so I’ve implemented some pruning so that users can have only 600 messages in their inbox/sent mail - this is all that the UI allowed them to see anyway so there will be no noticeable effect to them.

My improved logging also identified a missing index from my bounce table (which contains all bounced email) so I was able to speed up queries there very substantially.

I’ve also done a review of the MySQL status and it turned out I needed to increase the table cache a bit, but after boosting that from the default of 64 to 128 and then 192 I am now seeing that ‘opened_tables’ is sitting at a constant value.

The major problem that I am left with is the messaging system. It is full of joins, it gets a lot of write traffic, it’s a very large table and is the main contributor to the slow query log. Unfortunately I don’t really have a very clear idea of how to sort this out yet!

June 9th, 2008

Denormalizing the tables

My latest project is to denormalize some of the heavily used tables. The core ‘person’ table which is used for displaying profiles is fine but the message, wink and friend invite tables usually need to be joined with the person table.

I’ve found that joins in MySQL are very expensive, the slow down is in the region of hundreds or thousands of times.

So I am going to amend the message directory so that instead of simply using a foreign key to reference the person who sent or received the message it will actually contain the basic details of the person. This will remove the need for the join.

It will give me a problem if people change their username, person type or names but presently only the latter can be changed through the web interface and if people do change their names then they will just have to accept that messages sent may have their previous names on them - which is just what happens with ordinary email anyway.

At the same time I have also decided to re-organize the photos directory. At the moment there is a directory for every member with uploaded photos in the top level which means I have a single directory containing 16,000 photo directories. To be honest the performance of this on ext3 isn’t bad but it’s clearly not going to scale. Under the new regime I have a maximum of 10,000 entries per directory.

I hope this works out as well as the summary tables has. Since I made that change the member home page has disappeared completely from the log of slow pages (slow = more than 400 ms).

June 2nd, 2008

UK Swingers Site Rankings for June 2008

I notice that Alexa have had quite a major change of their algorithms, instead of just using the Alexa Toolbar they are now using data from ISP’s which brings them into line with Hitwise, Compete.com and the other big players.

It’s great news for us because it’s really good to have more accurate rankings. The bad news though is that just after they introduced the change we took a dip! But overall it’s not really resulted in much of a change.

Alexa Rank PageRank Site
37 7 AdultFriendFinder
3,474 4 Sex in the UK
13,113 3 Swinging Heaven
17,642 4 SDC
42,809 3 The Adult Hub
53,662 3 Fab Swingers
83,065 2 Local Swingers
87,841 2 Dogging.co.uk
106,885 1 Swing Fans

So our friends at Swinging Heaven continue to fall, The Adult Hub has also dropped slightly and Fab Swingers has risen slightly.

Meanwhile at the bottom of the table Dogging.co.uk has dropped quite a bit but the surprise has been Swing Fans which has dropped so far that it is heading for removal from the table.

I also had a quick look at members online earlier this evening, Fab Swingers and The Adult Hub were equal and Swinging Heaven had four times the number of members online (last time I did this check they were nearly seven times bigger).

The results from Google Analytics have also shown a pretty steady increase in members, page views and visits so overall I’m pretty happy with life!

May 31st, 2008

Badware Warning

The problem du jour was that we ended up triggering Google’s Badware filter which meant that anyone visiting the site from Google got a warning that the site contained badware which might damage their computer. Needless to say pageviews dropped a bit!

I logged in the Google Webmaster Console as soon as we got complaints and checked it out. It said that the homepage was an example of a page with a badware problem. The only two external links on the homepage are to Adult Friend Finder (via my affiliate account) and a banner ad from a very reputable network ran by one of the largest companies in the world.

So I complained to Google and within an hour or so they’d removed the badware warning, so overall I’m quite happy with their response.

May 10th, 2008

More cache work, summary tables and video upload

The interesting about Fab Swingers is that virtually all of the membership comes from word of mouth recommendation. So really the best way of getting more members now is just to work on improving the site (except for regions where we don’t yet have a critical mass of members).

The past couple of days I’ve been working on scalability. Profiles are now cached (with memcache) in a better way. I’ve also removed a whole bunch of counts in SQL and replaced them with summary tables.

Finally, I’ve also been working on a new feature. One of the things that we’ve really missed is video upload and someone requested it in the swingers forums. I’m not going to do this myself, instead I’m going to integrate with a video hosted service but it should be fairly transparent.

None of this has been deployed to the live server yet and there will be a short outage too — it’s going to take a little time to build up the summary tables.

May 10th, 2008

The Case of the Missing Page Views

I was just looking at the site earnings and I got a bit of a shock, from May 1 to May 4, pageviews have absolutely tanked by about half.

I went straight into Google Analytics and found that visits were holding up fine but the problem was that the bounce rate had rocketed from the usual 3.5% up to 12.5% and the page views per visitor had also fallen away.

So it looks like there was some sort of technical problem which put people off using the site for about 4 days and then everything went back to normal. It wasn’t such a critical problem that it killed the traffic completely but maybe the site was just slow or perhaps people with certain ISP’s were having a problem.

Having said that, we had no email reports of problems and usually as soon as there is the slightest glitch the support email box is filled with hundreds of complaints within minutes.

I’m not sure there is much more that I can do to investigate now so I’ll just park it for now and go back to it happens again.

May 1st, 2008

COB

I’ve just finished my last day at work and returned from my leaving night out (so excuse any typos!). My plan from now on is to make a living from Fab Swingers and the other website I run.

It is quite a big step but it’s something that I really needed to do. Both websites need more time spent on them if they are to realise their potential and I was just being over stretched by trying to do too many things at once.

It’s also a bit more pressure though, as my hobby turns into my job. One of my friends has a great expression for this type of situation: COB which is Cock On Block!

Search

Archives

Categories

Blogroll

My Advertisers

Resources

Favourites

Subscribe

RSS

Add to Bloglines

Add to Google

Add to My Yahoo

Add to My MSN

Add to Newsgator