I’ve uploaded my new book to the interwebs so you can now download and read through it, if the fancy takes you, completely free of charge (SEO Truth – A Bible For The Next Generation Of Search Engine Optimisation).
There may be a few blank pages because it’s been laid out for print; you can get yourself one of these hard copies from this website here.
Let me know what you think and submit some feedback over on lulu by all means! Cheers.
Edit: The download link works correctly now, oops.
As usual I’ve been spending a horrendously long time without writing anything on my blog – and for that I apologise. However, I have spent some of my time writing an SEO (Search Engine Optimisation) handbook, covering the importance of next generation techniques and practises.
I’m sure there are those of you who are all too familiar with the increasingly backwards approaches used by a few ‘special’ SEO agents and individuals out there and perhaps for you this will merely reinforce what you already knew to be true. For those of you who don’t know what I’m talking about -then please read the book and have a good laugh at yourself for being such a silly.
You can order print copies of the book – just not yet… more details on that coming soonly! I’ll be publishing online chapter by chapter (honestly I have finished writing it, but as an SEO, if I didn’t serialise it then it would look bad).
Enjoy the read and let me know what you think, if the first edition is terrible and you order, of course it’s going to be valuable in 200 years!
First off, I’d like to introduce myself. I’m a Search Engineer, a developer and programmer. I’ve worked with clients throughout the advertising industry at many different companies. My specialty is developing software that works with the search engines of companies like Google, Yahoo and MSN and attempts to influence the rankings of my client’s websites, as well as report on those ranking changes. I’ve never been to a lecture on computer science, read a book on development methodology and yet I’m in demand. My skills lie in understanding the technology of a search engine and how to capitalise on their ranking algorithms, web crawlers and content filters and it’s the ideas I generate in this area which have kept me in gainful employment.
SEO (Search Engine Optimisation) used to be a fairly simple task where you’d make sure every page on your client’s site had Meta tags, descriptions and content unique to that page. You might then try to analyse the keyword density of your key terms to keep them somewhere between 4 and 7 percent. More often than not most SEO companies wouldn’t even attempt that.
What most SEO companies would never tell you, and this is the industry’s most well kept secret, is that they’re intrinsically lazy. If you had a good client, with good content and a product of interest then their SERs (Search Engine Rankings) would climb entirely naturally to the top spots, you’d have nothing to do but sit back and reap the benefits of your lack of work.
This is of course a sad state of affairs which no real SEO company would allow and part of this book will help you to spot the difference between a professional outfit and rank amateurs and define the widening gap between the two camps.
As the title suggests I’m writing about the next generation of SEO. It’s becoming more difficult to increase the rankings of a particular website and it will only get more difficult to manipulate a website’s ranking without any understanding of how new search engine technology works. Lucky for you, my field is semantics (how to correlate the relationship between one word and another essentially) and you’re in for a whole chapter in manipulating a semantic index similar to those increasingly used by the major search engine players.
Chapter 1 – The Past
In order to proceed correctly in the future, the most important lesson is for us to understand what happened historically. There’s no shortage of information on the internet and amongst SEOs and webmasters about how Google’s original PageRank system worked. This is in large part thanks to a paper written by Google’s founders, Larry Page and Sergey Brin, whilst they were still studying for their PhDs at Stanford University. Not long after that they received their first investment from a company called Sun Microsystems which enabled them to build upon the hardware they had in their university dorm room and create the international phenomenon we know today.
PageRank was essentially a very simple system. It counted each link from one site to another as a vote for the destination site. By voting for another site the original gave away some of its own PageRank. The idea came from Salton’s Vector Space Model, which is a mathematical principal known to most Computer Science graduates today. This simple method of calculating which websites had the most votes, and therefore deserved higher rankings, is key to all search engine algorithms as it’s extremely fast to calculate. The most important factor in any search engine is its speed in returning and ranking results, especially when you’re dealing with an index of billions of pages.
The Anatomy of a Search Engine, based on the work of Larry Page and Sergey Brin whilst at Stanford.
If you understand that all calculations undertaken by a search engine must be as fast as possible, it allows you to draw logical conclusions:
· Thinking about a page as a machine would (which struggles to actually understand rather than just read), rather than as a human, is key to analysing your websites content for SEO value.
· Is every single underlined heading, keyword color, font size, image location, keyword relationship and page title length analysed when a page is crawled? It’s highly doubtful that anything too in depth is going to be indexed, when the crawler has another hundred thousand pages to visit and rank as quickly as possible, use some common sense here. Of course as processor speeds and bandwidth increase more in depth analysis will become possible in a shorter space of time.
· The search engine needs to maximise two things: the speed of its calculations and its measure of quality relevancy. Occasionally one is going to suffer at the importance of the other, if you were going to choose between indexing a page poorly – or not at all – which would you do?
SEOs in the past were able to capitalise on this speed issue by choosing to concentrate on areas of a page such as the Meta tags, description and page title. The content itself gradually became more important as time went on but still was subject to the speed of indexing. SEOs quickly realised that keyword density (how many times a keyword appears on a page out of the total number of words) was a very quick way to determine some kind of relevancy, and that the search engines were using it too.
Once the search engines got wise they implemented filters that stopped SEOs from flooding a page with keywords. Arguments in the SEO community followed over exactly what was the ideal keyword density for a term, and this usually settled somewhere between 4 and 7 percent.
Of course the PageRank model meant that agencies were keen to build as many links to their client websites as possible. To make matters worse however they were after links that already had high PageRank values to gain the maximum ranking as quickly as possible and this sprang up a cottage industry of people generating high PageRank links, purely to sell on. Google of course were unhappy about this and their anti-spam team began its work. Blacklisting of websites which ‘farmed links’ was becoming fairly common and this moved on to other aspects of ‘black hat’ SEO behavior – where an unfair advantage was being made by some nefarious companies and individuals.
Most SEO agencies at this stage relied heavily on staff who’d be subjected to some extremely tedious and repetitive labour. Going through page after page of a website and adjusting the number of keywords on a page, slightly changing each page title and Meta tag was a boring job and not well paid.
Directors and CEOs didn’t have a whole stack of problems though, if they kept building up link relationships with ranking websites and making sure their Meta tags were in place, their job was done. Often enough they’d have clients who already had an interesting product which did most of the work itself, spreading links around the internet as people registered their interests.
This natural traffic increase was what Google was looking for as they wanted sites which progressed on their own merits rather than trying to beat the system.
I’ve said many times before that so-called SEOs out there need to stop believing every word that spills from Google’s overactive pen. Google is a business just like any other and they feed information that’s deliberately misleading to stop people from gaining an unfair advantage with their search rankings.
It now appears that, in fact, Google assigns an estimated worth to each ranking on their pages – visible to members of their AdWords sales team they use the information from your PPC campaigns and analytics package in order to figure out whether you’re worth it.
I’m sure many of you will draw your own conclusions from this and in time we may see a Google press release, from that department which again knows as much about how their technology actually works as most of the SEOs do. Take it with a pinch of salt is my advice and invest the time to understand how a search engine really works.
This story was broken on the french blog Zorgloob, much credit to them for a brilliant find.
One of my favorite blogs, that I read just about every day is readwriteweb, a sterling tech, web 2.0 and search blog. Not so long ago their AltSearchEngines regular article was turned into a fully fledged blog in its own right headed by Charles Knight who knows about the existence of more search engines than probably anybody else on the net.
I checked it out this morning and spotted an interesting article:
Today we launch Part I of our 3 Part Series
Part I: What is a Search Engine? by Nitin Karandikar (Mon)
Oh glominy! I thought, glibbily. This is right up my street so I settled in for a powerful, thought provoking read.
Alas, the writer was a complete nitwit and I felt compelled to post this raging comment:
You’re completely wrong, I don’t know why on earth you’d try to reclassify what a search engine is when we’ve known what search engines are for a long time.
A search engine is simply “an information retrieval system designed to help find information stored on a computer system” (Wikipedia).
1. It enhances findability of relevant web content for the user
It doesn’t need to have anything to do with the web. Findability is not a word, even in italics.
2. It searches the entire web or a large subset thereof
(this excludes publisher search engines that search only a single site or group of sites)
No search engine searches the entire web. Don’t listen to the Google PR machine so much, and again, it doesn’t need to touch the web to be a search engine. Plus you’re on AltSearchEngines here… how many verticals do you guys cover?
3. Searches are specified using a keyword, phrase or question, or using input parameters, without the need for undue navigation
(I don’t consider pure directories like dmoz to be Search Engines)
So you’re saying you need an input to get an output? That’s genius.
4. It provides search results on demand, not periodically
I don’t even know what the hell you’re trying to say this for. It’s still wrong. Why does it have to do as a person asks it?
5. It provides some kind of unique or special processing of its own: either in the search algorithm, or in UI improvements, or both
(this excludes pure Rollyo or Google Coop-based search engine subsets)
This is far and away the worst thing you’ve written, you’re clearly grasping at straws. That is until you said:
The criteria described above will not remain static; as technology progresses, Search Engines will need to support increasing levels of functionality to be taken seriously.
No, i’m afraid a search engine, will always be a search engine. No matter how technology progresses it will still be a search engine.
The article you should have written is, “What search engines should have on my holidays”.
Yakov: A search engine doesn’t need to have its own index of the web or build it. A crawler of some description is responsible for building an index – that can take many forms and is often included in the search engine software itself. If you want examples of search engines without their own index, then take a look at the recent Digg API contest for some examples.
I’m hoping Charles gives you a massive kick up the backside and stops you writing what essentially is a load of bollocks.
Yes, it was a little scathing, but I get extremely irate when I see article written by someone who clearly is just trying to write for the sake of saying something. Especially on a source I have a lot of respect for because I don’t want to see them letting it through to the front page, that’s their role as editors – to weed out the rubbish and go with the quality content right?
Ignore the numbers in the title for the moment if you will and focus on these keywords: social, networking, search, community.
Web 2.0, by many definitions is all about allowing users to network, interact and the read/write web. Search 2.0 in that context does not yet exist. There are in some instances communities that happen to be built around a search engine such as Yahoo and there are new semantic search engines that let the users tag pages and documents to be found (something I’ve talked about before and pointed out as next to useless). None of these let the users actually interact with which results are returned. There is no networking or interaction that takes place with the search engine itself and this is just plain wrong.
Do you know how many people are on the internet at any one time? I sure as hell don’t but it’s a big number 🙂
Who creates all the content that ends up on the internet anyway? It’s not machines, it’s people, human beings are ultimately responsible for all the content on the internet and that’s never going to change. So why are we asking machines about content created by fellow humans when most of the humans are online anyway and know far more about their subject, and where it’s covered on the internet than any machine is ever likely to?
Enough rhetorical questions, I’m going to tell you what the real Search 2.0 is and you’re going to shout at me and tell me I should have patented and I’m a fool. However, if you don’t hire me to build it for you then you’re a fool because I get these ideas on a daily basis and I will crush you at some point in my life. Just kidding, I’m a fan of open ideas as well as open source especially when they’re for the benefit of us all.
- An instant messenger application or website with a live AJAX interface forms the centerpeice of the front end.
- Users create accounts and select their areas of interest by entering specific key phrases for those topics they feel most knowledgeable about.
- Users can then also select web pages that match those highly specific key phrases if they choose to.
- The search box appears as normal, you enter your query and the fun part of search 2.0 begins.
- Your query is analysed against users on the system, what occurs at this stage is actually a search for users with the best matching key areas against your query.
- If these users are online they can respond directly to your query, either suggesting a web link or entering a chat with you.
- If no users are matched online, then the suggested web pages are searched for the best matching content.
That’s the basis of it, but let’s have a look at the immense social power here.
Firstly, you get to rate the responses you receive, meaning that people can gain a reputation score for specific subjects and topics giving them an online credibility for that topic.
Sorry we just had an office fly-hunting session. Don’t ask.
Right, where was I? This system is by its nature, very low spam, it can’t be manipulated to provide results that are less useful because if you try and peddle a corporate product that’s crap, your reputation will drop very quickly and you’ll be banned. If the product is good on the other hand then who’s going to mind being directed to it if it answers their specific need and that’s better advertising than any money’s going to get you.
This concept is all about the users, no massively complicated algorithms need writing here it’s just using the very advanced and articulate knowledge of the very people who create the content you’re looking for, and to get the best answer you’ll ever get from a search engine is it not worth answering a couple of questions every now and again about the subjects you enjoy?
You can also bookmark people just like in any other IM and make friends with people holding the same interests, who you’d never meet on any other social network, and certainly would never think to find from a search engine.
I’ve previously posted a video of the new search interface I’ve been experimenting with. In order to get some feedback on how it works I’ve put together a small application that uses the Digg API to display the popular news stories on your desktop background.
The application will update for the latest stories every two minutes or so and refresh the tiles accordingly. If you mouseover a tile the window will ‘fisheye’ slightly, similar to the OSX dock. Click on the tile to expand it to a readable size and then if you decide you want to go to the story then double click the expanded image to open it in your web browser.
You can close the application by right clicking the little rss icon in your system tray and hitting Exit, or if you encounter problems then close the process newsview.exe in your task manager.
You will need the .NET2 framework to run this and the installer should point you in the right direction if you don’t have it. Failing that then go here to download it manually.
I stress this is a little alpha level experiment, but if you do encounter problems then I’m only too happy to help, just leave me a comment here and I’ll try and fix any bugs you may find. What I’m really looking for though is some feedback on the interface: Is it simple enough? Can you see the result clearly? Would you use a search engine that delivered results in this manner?
To download ‘DiggTop’ click here and enjoy.
Ever since I was five years old and we had our first home computer – a BBC – I’ve been programming.
No one ever taught me how, in those days I would spend hours trying to learn how the processes worked, happy to sit and debug until I understood.
At school I was no great mathematician I admit but I still spent all my time outside of school programming as much as I could, being fascinated with computer games and the internet in general. I learned that the math happened in my head, and my ability to learn new languages and read patterns was very useful. My first websites in html, and then PHP were shambolic but I persevered despite going to many schools, my parents were in the RAF so travel was a major part of my life.
At university I was unimpressed with my course – with exams on how to use Microsoft Word, I realized my potential wasn’t being reached and I left with the offer of a position abroad in Cyprus on a long term contract.
When I returned I was surprised to find I was in demand and felt in the right place for the first time as a professional when I joined an advertising agency’s new ‘E-Communications’ department as developer ahead of hundreds of other applicants with degrees and more experience in industry. Since then I’ve worked on websites and advertising for companies from Walmart to Sony and across many different market sectors.
In the UK, I can now go for any position I want and feel confident I’ll get it, I turned down a position at a major search engine in order to take my current job. That’s because they can push me and use my to my potential. In the UK there are no jobs in search related fields that aren’t SEO and I don’t want to sell my work to a company who won’t use it effectively.
I do enjoy it, being able to work with designers and with an energetic team enables me to work as hard there as I have everywhere else. I strive for perfection in my work and enjoy the social life – I’m not a typical nerd as defined by stereotype. I’m keenly competitive as a sportsman.
I have ideas I can’t implement, I write down implementations that will never be used and concepts I will never have a response on.
When people ask me; ‘isn’t programming boring, how do you cope’, I tell them that programming is an art to me, it’s not just science, with an end result that can be beautiful and stunning.
This is what I look for in my work, this is what motivates me to write, I need an output for all the ideas that may never happen. People accuse me of not knowing about search, or being too critical of search companies but understand; this is my passion.
I welcome criticism myself, it lets me grow and adjust my ideas and without it what compass would I have. Look around this blog you’ll find comments calling me a f**king moron. Why would I delete a passionate response to an article that I wrote from the heart?
In the coming months, my first commerical search engine, running entirely on your own PC desktop will be released. It has features you’ll have never seen before I promise you that, it may well revolutionise the way you search for documents, web pages and folders. Rather than telling me it’s crap and I’m a f**king moron (if that’s how you feel), it’d be great if you’d tell me why you think that – and how you’d change it. Then I promise to answer you in kind.
I have a lot of respect for the readers who don’t like what they see, because I feel the same way when I write about a lack of innovation in search. If you don’t criticise what you love, it’ll stay the same whilst you as a human being never will. You’re changed by the criticism, praise and ideas of those around you.
Please, keep trolling.