What kind of mistakes could these “data inputers” have made? It’s best illustrated by an example…

If Alice was taking down a customer’s email address, say Reginald@hotmail.com they could have typed the following…

• Reginald@@hotmail.com – typed a single character or letter twice
• Reginald@htmail.com – neglected to type a character
• Reginald@htomail.com – typed all characters but in the wrong order

Having noticed this problem in BrightMinds’ database I wrote a TSQL script to identify and correct these human errors.

The script reads your existing email database and identifies domains whose emails have recently been read. These are inserted into the table domainKey. Next, these domains are read sequentially in a cursor and the human error domains created. A reference table is then created – domainSynRef with two columns – realdomain and errordomain.

Of course some rearranged domains could well be real domains, the script checks for this but could do with some development. Clearly if anyone else wishes to use the it will need some light table name or structure modifications. The basic string manipulations though are still entirely valid.

So here’s an example for what it generates with the domain aol.co.uk…

Quite comprehensive I think, could it be edited to introduce double errors such as Reginald@hhotmail..com? Sure, if you then write some more string statements after the cursor. Although there’s probably a stage when one has to think – if someone’s made this many errors, can we assume we can fix it?!

As always, suggestions/modifications/improvements very much welcome. I imagine a single run script and subsequent checking function would be better for a single email address input and would probably work better than my stored procedure, this is just what fitted best into our environment (checking c.300,000 email addresses).

The result? We gained over 1500 now valid email addresses. Try it out for yourself, download the stored procedure here.

Unfortunately there’s no easy “X number of steps” way to gain instantly great email sending reputation with ISPs. What’s required is to follow massive amounts of guidelines in your email processing, design and send stages with strict attention to detail.

So to help us all I’ve put together a broad but detailed 7 step guide to gaining great email sender reputation. It’s a compilation of all major ISP and webmail provider’s bulk mail sender guidelines and will, if executed fully, lead to very good sender reputation.

Design & Setup – Before you start sending

1. SMTP Mail Server.

When building your SMTP Server, whether using Windows or UNIX/Linux you’ll need to ensure it’s on a separate, fixed IP address to your business and transactional emails. You’ll also need to setup a reverse DNS record against its domain. E.g if your fixed sender IP is 89.98.252.65 then ask your ISP to setup the reverse DNS record email.yourdomain.com and ‘point it to’ 89.98.252.65. The SMTP server also needs to send authenticated emails using the SPF/SenderID and DKIM/DomainKeys frameworks. A few months ago I wrote a post describing how to integrate DKIM/DomainKeys into Microsoft Exchange Server. There are plenty of guides out there explaining how to set both of these up on UNIX/Linux.

2. Clean Lists

Design your newsletter subscribes process to follow a ‘double opt-in’ mechanism to minimise junk/wrong/spam-trapping email addresses. In other words ensure that when someone signs up to your service, send them an initial email that contains a verification link. When clicked the user’s email address is verified and added to the mailing list. This stops us trying to send emails to fictionaladdress@hotmail.com and punishing our reputation when we receive a hard bounce from hotmail.com. Don’t know what a hard bounce is? You’ll want to learn about them.

3. Unsubscribes, bounces and bad formats

Double, triple and quadruple check that your system processes unsubscribe requests quickly and correctly. Likewise for soft and hard bounces; ISPs are well known to assess sender credibility heavily on the number of bounces they generate. Lots of bounces means a poorly compiled or maintained emailing list.

If you collect email addresses by phone, post or acquisition make sure you have some form of bad format checking in place to prevent typical errors like barry@hotmail..com or barry@@hotmail.com. If you want to get extra fancy, script it to dynamically replace historical (pre double opt-in) email addresses like homail, ohtmail, hptmail with hotmail. I’ve written my own in TSQL and will publish it shortly.

Monitor your new subscribers for spamflag addresses – abuse@, postmaster@ and nospam@ addresses should all be automatically unsubscribed.

4. Sign up to email feedback loops (FBLs) and whitelist services.

FBLs simply notify the email sender when the recipient clicks on the ‘This is Spam’ button while reading your email. Since these recipients have deemed our email to be spam (and the ISP concerned knows this) we need to unsubscribe them to ensure we build good reputation. Many of you will have signed up with the big ones AOL, Hotmail and Yahoo but do make the very small effort in signing up for the rest. Word to the wise maintains a good list of all known feedback loops and whitelists along with a script to make processing their spam notifications a little easier. If processing these FBL notifications becomes a drag be sure to read my solution on how to automate it.

Starting up – When you start sending

5. Test your messages

Use design tools such as Email on Acid and Litmus to test your designs in multiple email clients to ensure they all look the same. Also paste the HTML into content checkers to get a “spam score” as well as suggestions on how to avoid the spam filters. SpamCheck and Contactology are both pretty useful.

6. Start small and often

Nobody wants to hear this, especially when they’re planning on sending 250,000 emails a week but it really is very important for those first few weeks. ISPs are fully aware of your sending trends and will consider a sudden increase in sending rates highly suspicious. Try to send small to begin with and slowly raise the sending rate.

Try also to be consistent with the times you send your emails. This may be difficult, especially if you occasionally respond to current affairs or the weather etc. Sending zero emails on Monday followed by 50,000 on Tuesday looks like a potential email server hijacking and may result in your mail servers being temporarily blocked by ISPs. Is there anything wrong with some people receiving ‘Monday Newsletter’, some ‘Tuesday Newsletter’ etc? Of course not, in fact depending on seasonality, staffing levels and demand it’s arguably quite appealing.

7. Design well

Follow CAN-SPAM requirements. It’s a complex list of requirements laid down in 2003 that were meant to stop SPAM. Clearly it failed but for us non-spammers that simply want our messages read by our subscribers we need to comply. OpenMoves is a decent post outlining the basic requirements.

Send Multi-part MIME emails rather than just HTML; ISPs look at simple HTML emails as likely manifesting covert “Click here for FREE Viagra” banners. Likewise, low text/html ratios look suspicious so ensure you pack a great deal of text into your emails. Product descriptions bulked out with product reviews works very well.  Avoid using attachments and keep your “From Name” and “From Address” consistent. Never link to another domain or port from within your emails.

Design for when images are enabled and disabled and use your imagination to think about how best to push your message. If you use a broad image in a table then don’t just use alt text, place <font> tags around the image to enlarge the alt text and make it unmissable. Look at the images below – which one do you think will get more clicks?

Rubbish, bland alt tags

Vibrant Alt Tags

Beyond the technical design, good, well thought out and tested design results in better customer engagement – better open rates, click throughs and conversions. Importantly though it also results in far fewer unsubscribe requests. Ensure you send genuinely engaging emails. Weekly “Hey look at our samey products that we’ve sent you a million times before” will pull in reliable revenue but you’ll likely see the clickers and converters are not your most recent or valuable customers. Likewise, people won’t be engaged if you just talk solely about your products with wholly one-sided conversation (there are curious exceptions).

Where’s this leading? Well webmail clients compile and analyse email open rates and click through rates by domain and judge your spam content on this. If 80% of GoogleMail recipients open an email from yourdomain.com then GoogleMail will deem yourdomain.com to be highly engaging and so decrease its likelihood of ending up in the Spam/Junk folder. If only 5% of them open the email, GoogleMail can assume that what you send is not really worth cluttering their inboxes. The lesson here? Only send your emails to those that want to read them.

Keeping that reputation

As already mentioned, try to remain consistent with broadcast levels and times. Does sending 50,000 emails on Monday and 10 across the rest of the week look suspicious? Yes, very, so stagger the send. The time of day you send is more likely to influence the open rate rather than directly affect your reputation. For example I know that most of my emails are opened at about 9.30pm, even if I send them at 2am. So I can achieve a better open and click through rate by sending them at around 9pm. Better open and click rates mean better reputation. For other businesses, especially B2B, it’s likely to be different.

Should you send all your marketing emails from one IP? Yes, to begin with at least because it simplifies IP reputation assessments by the ISPs. However, once you’re up to speed it’s worth sending through a second IP via another ISP. Imagine if your ISP suddenly changed your IP range, went bust or just felt like being irritating – you could lose your IP and so all that built up good reputation – disaster! So like with the first IP, create a reverse DNS record and begin sending low volume. Slowly build the volume and split the sending across both IPs.

Monitor your deliverability across domains by collating the number of emails sent and comparing it to the number of emails opened by domain. This, monitored weekly, gives a great view into your deliverability by domain. To increase the definition, look at the same data over a single day. This will reveal whether you’re reaching short term sending limits with some domains. To check whether you have any long term issues, be sure to check MX Toolbox’s Blacklist Checker.

Sending emails is not an easy exercise to get right but as with so many things in life, if we tick all the boxes and follow the correct guidelines, we can create really very successful email marketing programmes.

If anyone feels like contributing to this then please email me and I’ll add your suggestions!

However, a problem arises when you send 150,000+ emails per week. You can then expect to receive around 200+ FBL emails per week, a frustrating unsubscribe routine. This is made worse by the completely informal FBL email structure. Yahoo! send their FBL email with a text file attachment that lists the recipient. Hotmail however just attach the original email in .msg format. AOL though are the most annoying – they attach the original .msg like Hotmail but strip out the recipient information and replace it with “<redacted>”. In the FBL body they state the original messageid, hence you need to query your SMTP server for the recipient’s email address. The inconsistency between FBL implementations just means it all takes up more precious time and it needs to be automated…but can it? It can! And I’ll show you how I did it.

Firstly, a brief outline of what I do. I send emails using Arialsoft’s Campaign Enterprise mated to SQL Server 2008 with Exchange as the SMTP server. I use Outlook as my general mail client. All run on Server 2008/Windows 7.

The FBLs are sent from Hotmail etc to abuse@ and processed by an open Outlook session to append their attachments .dat/.msg/.txt {etc} to their message body. They’re then read by a terrific application called Email2DB by Parker Software which reads the email body and extracts recipient information. Email2DB then runs a query against the SQL Server to unsubscribe the recipient. Email2DB is not free, however once you get to grips with its interface you’ll realise that you can perform some pretty funky things with it and well worth the price.

So where do you start? Assuming you’ve signed up to (and configured/confirmed functional) all the email FBLs…

Firstly, to get around AOL’s annoying <redacted> feature, alter your email templates to include the recipient’s email address somewhere in the body. For example, amend the footer to read something like ‘This email has been sent to {email_address} because…’

Now download and install Outlook Redemption – this allows us to run VB scripts within Outlook that can access the data within .msg format attachments. To get things up and working get the developer edition.

Next setup a rule in Outlook that moves emails received from the FBL addresses to a specific folder – “FBL Queue”. Now setup a further rule, beneath the previous rule, that performs a script. Use the script below, you will need to set Outlook’s macro security to low to run the code.

AppendEmail.vb (v.0.1)

The code, if you don’t know VBS, pulls out the attachments and stores them in a temporary folder. It then reads them and appends them as plain text to the original email body. Inside the message body’s are of course the recipient’s email addresses.

Now download, install and start-up Email2DB.Set it up to store the messages however you wish, it doesn’t really matter how, I chose to use the internal database. Setup an account called “FBLs” and point it, using MAPI to read your FBL Queue folder. Set it to move read emails to another folder called “Processed FBLs”. Now we need to setup the triggers.

Setup a new trigger called “Hotmail” and set the “From address” to “staff@hotmail.com”. Click on the tab called ‘Field Extraction’, click ‘Add Field’ and paste a copy of your newsletter/triggered email HTML in the textbox on the right. Now, in the textbox on the left titled ‘Look For’ type in the text preceding the recipient’s email address (red in the example below). Then in the ‘Extract Data’ tab, ‘Until These Characters’ box, type in the text following the user’s email address (left in the example below).

“This email has been sent to example@emailaddress.com because…”

If you now click on the General tab the email address should be highlighted, indicating that it will extract that data. Click OK to go back to the Hotmail trigger and OK again to go back to the trigger icons list. In exactly the same way setup further triggers to extract data from AOL and Yahoo! FBLs only remembering to change the from address for filtering to scomp@aol.net etc.

Once they’re setup, go back into each trigger and ensure the ‘After this trigger has processed a message’ is set to the following trigger. So for example, once the ‘Hotmail’ trigger has been executed, it follows on to process the ‘AOL’ trigger etc.

Now we have a system that executes a script on each FBL entering the inbox, appends its .msg attachment to the message body and extracts the recipients’ email address. All that’s left is to setup Email2DB to do something to unsubscribes people!

In my SQL database I simply insert the records into a table called fbl_unsubscribes with fields email_address, fbl_domain and dt_created. I then use a stored procedure within the database to change the subscribe list later. This means I can create historical reports to view abuse notifications by domain over time. Email2DB supports a wide range of database protocols, for details on how to deal with SQL inserts see here.

So there you have it – a 100% automated feedback loop unsubscribe system.

Limitations? My Outlook script is rather weak because it stores msg attachments as draft emails (required for reading). It could do with deleting them afterwards. It also requires an open Outlook session. Improvement suggestions always welcome!

Ignoring the expensive options to guarantee image blocking is switched off (Goodmail etc) we can only really do two things. Firstly, we can encourage recipients to disable image blocking on one of our emails early in the relationship. Once image blocking is disabled for the sender, that setting will normally be saved permanently. The other option is to take full advantage of Alt tags. These are the short blurbs of text or ‘Micro-copy’ shown instead of images when the images are unavailable or blocked.

Annoyingly, not all email clients/web clients actually support showing Alt tags (grumble). Here is a list of most current email clients and whether they support displaying Alt tags…

The important things to point out are that Outlook 2007 and 2010 do not show Alt tags and neither do any of the Mac clients. On the bright side though all the webmail clients do show them. As a random observation, isn’t it a little strange that whilst the Microsoft Hotmail Client displays Alt tags, Hotmail online does not?!

So here are my suggestions for getting your recipients to permanently disable image blocking…

• For one of your first communication triggered emails, try sending something with a tempting subject line and a body made almost wholly of 1 image. Include text above the image politely reminding the user to disable image blocking. Simple but some recipients will disable image blocking.
• Instead of using dull, informative Alt tags like ‘logo’ and ‘menu image’ try using ‘Disable Image Blocking’ for every alt tag seen in a typical email client viewing pane. This is clearly a powerful call to action and I’ll be running a randomised test of precisely this very soon. I will post the results.
• Remember to think long term and ensure you write your Alt tags to get your recipients to do what you want them to do. If you write ’50% off’ you’re clearly more likely to get a click than a disabled image block. Do you think ‘unbeatable offer image’ or ‘disable image blocking’ would perform better? Which will provide better long term click through for future email campaigns?
• As ever – experiment, measure and experiment again.

Hope you enjoy it, 64 Diggs so far!

Sending emails used to be simple however then came the fraudsters and so in the interests of security, email verification and authentication technologies were introduced. These are Domain Keys, DKIM, SPF and SenderID.

Enabling these authentication methods is a significant and important step in helping our marketing emails hit peoples inboxes. See MailChimp’s page for a demonstration of how many significant domains rely on these different authentication schemes to verify email senders. It’s a great article.

So, lets start with SPF and SenderID…

Firstly, skipping the long explanation, they’re 99% the same. Very basically, when a recipient reads an email from yourdomain.com it cross references the originating IP with yourdomain.com’s DNS server to verify that the email server’s IP is allowed to send emails. Simple.

To get this working go to http://www.openspf.org and complete their SPF record creation wizard. You’ll end up with something looking like…

"v=spf1 ~all"

Email this to the company that looks after your domain and ask them to add it to your DNS server as a text record. SPF – done.

Moving on to Domain Keys and DKIM…

These are again very similar but differ from SPF/SenderID in that rather than simply verify sending rights, they dig a little deeper.

When sending the email, the email server attaches an encrypted digital signature to its header. The recipient then processes this signature by cross referencing it to a text record held on yourdomain.com’s DNS server. Hence the recipient can again verify that the server sending the emails is genuine and legitimate, a little more securely.

So, getting it to work in Exchange…

Exchange doesn’t actually support DKIM or Domain Keys so you need to buy an after market software package. AdminSystem sell it, and it’s a not-so-cheap $799 for the Enterprise version (allows a number of servers to use it for the same domain). It supports not only Exchange but also IIS SMTP server so you can use it in several environments. The product is here and called (aptly) “EA DomainKeys/DKIM for IIS SMTP Service and Exchange Server”. It may look like an amateur website selling a fake product but it really works!

Installation is simple: download the MSI, install it, restart and voila…almost. Mid way through you need to create the DNS record and luckily for all of us there’s a video on youtube explaining exactly how you do this. It’s here, try to ignore the terrible music in the background. There’s also almost a minute of pointless pause at the end so be prepared to ignore it.

Once you have the text record you’ll again need to send this on to your DNS host for them to add to youdomain.com.

Lastly we need to test!

For Domain Keys/DKIM, send go to Mail Radar and send an email to the randomly generated address at the bottom. After a few seconds click view results for verification.

For SPF/SenderID we don’t even need to bother sending an email, just go to PoliteMail and type in your email address. PoliteMail will then verify the SPF record.

Bingo! DKIM, Domain Keys, SPF and SenderID all on an Exchange Server!

Designing and coding an HTML email is not like designing a simple web page.

Ok so they’re both coded in HTML but the approach is entirely different. HTML emails could potentially be viewed in many different email clients – Outlook, Outlook Express, Windows Mail, Mac Mail, Eudora…the list goes on not forgetting the hundreds of online services like Hotmail, Yahoo Mail and Google Mail. All these clients and webmail services have their own little quirks which when compiled and compared means coding with any kind of modern techniques is near impossible…

• CSS in the <head>? Forget it.
• CSS at all?! Dodgy ground.
  
• JavaScript? Goodness no.
• Floating <div> elements? Are you serious?!
• <div> elements? I wouldn’t…
  

Outlook 2007 is what many cite as being a frustrating turning point in email client HTML rendering when Microsoft chose to switch the rendering engine from Internet Explorer (as it was in Outlook 2003) to Word 2007. Thus HTML rendering was crippled. Microsoft can’t take all the blame though, Google Mail refuses to interpret externally linked stylesheets and ignores numerous seemingly random inline CSS properties.

For a depressing comparison of Outlook 2010′s attempt to render to the same level as Outlook 2000 see below…

Outlook 2000 vs Outlook 2010

…this was, according to Microsoft, due to security concerns.

So, the question is…

How can we design marketing emails that look great in all email clients, webmail and offline?

Surprisingly, you’ve gotta code like it’s the year 2000, back when flashing text looked great and play.com was called play247.com (and looked like this). This means <table> elements with all their dated and inflexible <tbody>, <tr> and <td> delights. Those dark times when web designers would need to use <rowspan> and <colspan> to create a grid of elements that couldn’t overlap.

My best practise tip is to physically draw a rough content grid for the email by pencil. As an example, below is an M&S email from January 2009.

M&S Marketing Email Jan 2009

The images are likely designed and sliced in Photoshop for convenience. Here’s an approximate HTML table grid…

HTML Layout Grid of M&S Email

Different tables are shown as different colours, with the white background representing a parent table. Each row is a separate <tr> and each cell is a table element <td>. COLSPAN is used to force a table cell to bridge two or more columns and ROWSPAN to force a table cell to bridge two or more rows. I’ve labelled the colspan attribute as ‘cols‘ due to shortage of space.

This email can basically be pseudo-html-coded as…

<table>
 <table>...table-elements...</table>
 <table>...table-elements...</table>
 <table>...table-elements...</table>
</table>

For a full tutorial carrying all details of how to build HTML tables as seen in this email I’d recommend this about.com tutorial. I would demo-code it myself, but it’s not even necessary as the full M&S email and source code is available here.

Once you create an HTML file with tables, table rows and table cells ensure you declare each table’s width and height where possible. It’s best to fill each cell with an <img /> and then link that image, otherwise it’s a potential wasted click i.e…

<td><a href="<link>"><img src="<img-link>" height="100" width="100" border="0" /></a></td>

Note: Always declare the image’s height and width in an <img /> tag. That way when the email lands in someones inbox the email actually looks readable/presentable before they click to ‘Allow Image Downloads’.

Generally, try to use solid background colours where possible as simple colours are better supported amongst email clients than background images. For cells where you want a background image and text content then use the background attribute. For example…

<td height="100" width="100" background="<url>">

This will enable the background to be viewable in all most webmail providers. Using an inline style may not (Google Mail). In fact because support is so patchy avoid using inline styles as much as possible.

When formatting text don’t use inline CSS styles, instead use the ancient <font> tag e.g.

<font face="verdana" size="1">Here is some Text</font>

If you use Google Analytics then you’ll have a good idea which font your customers have from their declared operating systems. As with all web design it doesn’t make sense to put your entire email in ‘Arial’ if 90% of browsers use Mac OSX.  It’s best to use a decent amount of body text to give spam filter’s the best chance of judging your marketing email as not being spam.

Generally, once you create a table/row/cell/sub-table/sub-table row/sub-table cell structure and fill it with linked/unlinked images and text then really there’s nothing else to it. Active content like videos are very poorly supported indeed and like I mentioned before JavaScript is a complete no-go.

For detailed and accessible documentation of which email clients/webmail providers support certain HTML/CSS properties then take a look at the Email Standards Project. It clearly demonstrates that whilst many web designers are thoroughly fed up with having to code for IE6, many email clients are just as patchy and awkward.

Once happy with your layout, design, and text then before you send anything it’s important to TEST, TEST, TEST and see how it looks in as many email clients as possible. You could try Litmus, an absolutely fantastic service that lets you see how your email will render in a majority of the major clients/webmail services. It costs $50 a month but it’s worth it.

For those of you that don’t want to pay for seeing how your emails will look in different clients then take a look at Email on Acid. It doesn’t have as many options or clients as Litmus and may be a little patchy on result consistency but it’s still great for a free service.

So to summarise here are the important things to do when when creating an HTML marketing email…

• Design the email as a grid by pencil first and try to keep the width to 700 pixels or less. Ideally 600 or less, the M&S one above was only 580px wide.
• Code the email as a series of HTML tables and sub-tables. Avoid CSS. If CSS is a must then cross check the Email Standards Project with your house file for recipient compatibility. Test Test & Retest.
• Use Photoshop and the slicing technique to create the images and have them fit your table layout exactly.
• Use COLSPAN to get a table cell to bridge two or more columns and ROWSPAN to get a table cell to bridge two or more rows.
• An image without a link is a wasted opportunity.
• Try to get a decent amount of text into the email so spam filters have an easier time passing it.
• Use <font> for text styling rather than CSS.
• Test Test Test your emails with Litmus or Email on Acid.
• Setting hex color values without a # symbol causes issues in IE based rendering environments (Outlook Express, Outlook 2003 etc). Thanks go to @dkurzius for that one!
• NEVER send an email from one domain and have all the links pointing at another domain. That’s what the spam filters want you to do!

Update: 14/01/2009

Email on Acid is nolonger free

However, it’s only $13 a month and so far better value for money than Litmus (who have been having some issues recently with webmail rendering). As such I’ve cancelled my subscription with Litmus. Sorry but EON is way faster and cheaper.

Double spaces in strings is a real pain. For those who can’t picture it see the example below…

A particular column in a row is declared as some kind of string (char etc) and the data one wants to store is something like:

'John Smith'

But due to a data entry/import error the following gets stored:

'John  Smith'

…or worse still…

'John    Smith'

This issue can be a real pain, especially when the data’s being used for marketing purposes. Nobody likes to receive postal mailings, especially when it’s addressed to ‘Mr John   Smith’.

The Solution

Firstly, the obvious will not work across all versions of SQL Server (notably 2000)…

select * from table where string like '%  %'

SQL 2000 and maybe 2003 will return every row with a single and double space. So instead we have to use the following…

select * from table where string <> replace(replace(replace(string,' ','<>'),'><',''),'<>',' ')

That will list all the rows with double spaces in column ‘string’ in table ‘table’. So to update the string and remove those bloomin double spaces run the following query…

update table set string = replace(replace(replace(string,' ','<>'),'><',''),'<>',' ')

Bingo!

Text files transferred by HTTP such as css, html, javascript etc are highly compressible – by their nature they can be compressed to a fraction of their original size quickly without much webserver CPU load.

IIS can perform this compression on your original uncompressed css/html/javascript source files and transfer the compressed content to people’s web browsers. Their browsers then expand the compressed files whilst browsing saving a great deal of bandwidth/download time. Important to note is that the people browsing are never aware that they’re browsing a compressed website.

Should I bother?

Brightminds.co.uk homepage download size shrank from 660KB to less than 400KB. This saving however includes image downloads and the real compression saving takes the page weight from 77KB down to only 12KB. Thus less than a sixth of the page size is downloaded by a browser at a tiny CPU load cost. End result? Since expanding the page on the user’s browser requires practically no CPU load, everyone’s browsing experience is significantly faster. But what about the webserver CPU cost? It’s far smaller than the CPU cost of uploading a further 64KB to the user.

The Bottom Line – HTTP compression is free and quick to enable. If you’re not using HTTP compression, you’re compromising your reader’s/customer’s experience and burning bandwidth.

Lets enable it!…

There’s a great deal of conflicting articles on the web about how to enable compression correctly in IIS6. Some say that you will need to edit the metabase whilst others say a simple cscript will do all. I’ve found you’ll need to do both and I’ll outline how below…

Make a backup of your existing IIS6 configuration by opening Computer Management, expanding IIS and then right clicking on it to All tasks – Backup/restore configuration.

Backing up IIS configuration

Click on Web Service Extensions within IIS, right click and select to add a new web service or extension. Call it HTTP Compression and add %systemroot%\System32\inetsrv\gzip.dll as a required file. Select to allow the extension and click OK.

New IIS Service or Extension

Within IIS right click on ‘Websites’ and go to Properties. Click on the Service tab and tick the box to ‘Compress application files’ (this will compress files sent with the extensions .asp, .aspx etc) and also tick the box to ‘Compress static files’ (compressing .html, .css etc).

Enabling Compression in IIS6

If you want to specify a different directory to store the temporary files to then do. Specify a storage limit, although brightminds.co.uk have up to 250,000 compressible hits a day I’ve never seen the compression directory exceed a few MB. 1GB will be more than enough. Click OK.

Now we need to specify which file types are compressed by IIS. Individually copy, paste and run the following commands into the Run box.

cscript c:\Inetpub\AdminScripts\adsutil.vbs SET W3SVC/Filters/Compression/deflate/HcFileExtensions htm html txt css js

cscript c:\Inetpub\AdminScripts\adsutil.vbs SET W3SVC/Filters/Compression/gzip/HcFileExtensions htm html txt css js

cscript c:\Inetpub\AdminScripts\adsutil.vbs SET W3SVC/Filters/Compression/deflate/HcScriptFileExtensions asp dll exe aspx asmx php

cscript c:\Inetpub\AdminScripts\adsutil.vbs SET W3SVC/Filters/Compression/gzip/HcScriptFileExtensions asp dll exe aspx asmx php

These commands enable the compression formats deflate and gzip for the specified file types. So for example the last command enables gzip compression for files with the extensions .asp, .dll, .exe, .aspx, .asmx and .php. You may ask why not compress .jpg image types and indeed all web content. Simple answer – text files compress quickly and exceedingly well, images/videos etc don’t.

Right click on IIS again in Computer Management and stop all IIS services.

Now Run the following command from the Run box to edit the IIS Metabase…

Notepad %systemroot%\system32\inetserv\metabase.xml

Editing the IIS Metabase

Find/search for the line that contains the following tag – ‘<IIsCompressionScheme’. There should be two of them, one referring to gzip and another for deflate (the two compression methods). HcFileExtensions should state the static file extensions we want to compress – htm, html, txt, css and js. Likewise HcScriptFileExtensions should state the dynamic file extensions to be compressed – asp, dll, exe, aspx, asmx and php.

What we need to do is edit HcDynamicCompressionLevel=”0″ for both gzip and deflate to be HcDynamicCompressionLevel=”9″. This defines the compression level on a scale between 0 and 10. 0 doesn’t compress and 10 uses far too much CPU load compressing, 9 is generally the best option.

Double check you have changed HcDynamicCompressionLevel for both gzip and deflate and save the file changes (File > Save).

Again in Computer Management Right click on IIS Computer Management and start all IIS services.

Your IIS web server should now be serving a compressed website. Double check it by entering your domain into http://www.whatsmyip.org/http_compression/.

Download and install the latest x32/x64 SQL native client.

Install SQL Server 2005 from your installation media and ensure that towards the end you select to NOT start the SQL service. Personally since I use SQL Server 2005 on my workstation as a test environment I’m happy assign it the Local System account.

Download and install SQL Server 2005 SP3…and just before it finishes installing you’ll see something like this…

SQL Server 2005 User Provisioning Tool for Vista

This is the SQL Server 2005 User Provisioning Tool for Vista and it lets you know that admin users on the Vista workstation (Windows 7 in our case) will not have admin rights in SQL Server unless explicitly stated. Close it for now since because the SQL service is not running, any new users will not be saved. The SP3 installation will now finish, abruptly.

Reboot your workstation to allow the SQL services to start.

From the start menu run the ‘SQL Server 2005 Surface Area Configuration’ tool. Select the “Add New Administrator” link and select which users you wish to assign access to. There is no confirmation, the dialog just goes away without any error messages.

Start SQL Management Studio and you’ll now be able to login. No errors, just a fully working SQL Server on Windows 7