Some Thoughts On Web Search

A buddy of mind recently started working at InfoSpace, a successful Web search company based in Bellevue. In conversation, he mentioned that they were the company behind dogpile.com, an aggregation search engine that queries MSN, Google, etc, for their results and then sorts them after some analysis. Being the kind of friend I am today, I figured I'd give them a run for their money by dogpiling myself.

The first result set came back a little different than I had expected:

At first, I was impressed that somehow Google thought to associate me with Linux training. However, my pride was quickly turned to horror as I found that most of the other top results were related to male performance issues! Also ironic was that the fourth result was for health.msn.com, but was found via Google Ads.

Figuring there must be a mistake, I figured I'd fire off another self-dogpile:

This time, no "tech training" love, just more adult-themed help offerings. I hit Refresh a few times to see if the results would change. After about a dozen refreshes, I ended up with one that had pure content:

I've come to three possible conclusions:

1. The first two letters of my first name are a common acronym for a condition which may require the help these results offer. Perhaps the dogpile engine and/or the engines it consumes make this distinction and assume that any request that contains "ed" wants unsolicited links to offers you can only find in your Inbox hundreds of times each day.
2. The dogpile engine is extremely sophisticated and actually knows that it's me searching for myself, so it assumes that I must need help in certain other aspects of my life. Either that or it's making fun of me, which doesn't make me feel any better.
3. Everyone gets these ads.

Regardless, I don't see using dogpile anytime soon. There's nothing that pisses me off more than a search engine that doesn't search for what I ask it to search for. The whole idea of searching for "ed kaim" (with quotes) is that I want results that contain that exact phrase. If you can't do that right, I can't even consider you. Yeah, I'm sure they feel they're "helping" me, but screw that. I know what I'm doing. If you think I spelled a word wrong, run the search I gave you and put a note in that I might have misspelled something. If you're right, I'll search again with your suggestion.

A reliable, helpful search engine is worth its weight in gold (which is why you can get a PE of 60 and still be considered undervalued).

I probably use Google to search the Web 100+ times each day. I hacked together a simple home page that provides a form for easily accessing google's search engine. It has two boxes (one for single words and the other for a phrase) and a submit button. My search process is:

1. Press Ctrl+Esc to open the Start Menu
2. Press I to launch Internet Explorer
3. Press Tab twice to select into the first input (words) or three times to get to the phrase input
4. Type the thing I'm looking for
5. Press Enter to execute the search on Google

Even better is that IE's autocomplete makes it very easy for me to scope results very quickly. For example, I can tab into the phrase input and type a period ("."), which will give me the three items I've used (".net framework", ".net compact framework", and ".net framework 2.0") so that I can press the down arrow a few times and get the whole thing included without having to retype. A few years ago I actually found that this model is better than using my local MSDN installation and have stopped installing MSDN since. The reasons are:

1. You get better results (not just the 400 places where an item was footnoted in an XML doc comment)
2. It's faster (loading an instance of IE and the local page is cheap vs local MSDN)
3. It uses less memory
4. It's always relatively recent without requiring maintenance from me

Yes, it doesn't work without an Internet connection. Fortunately, I almost always have one.

Anyway, I'm sure you can do all of this stuff in other browsers on other OSes. My point is that Web search is so critical to my job that I can't even consider an engine that doesn't search like I ask. I could easily hack up my home page to use another provider, but I haven't found one that does better than Google. Given that, I can't even believe that there are search engines out there that won't even search for what you ask for. Seriously.

11/7/2006 2:33:00 AM (Pacific Standard Time, UTC-08:00)      Comments [2]  

If You Have Thirteen Minutes To Burn...
Watching this video may be the best way to do it. Absolutely amazing.
10/29/2006 1:24:13 AM (Pacific Standard Time, UTC-08:00)      Comments [0]  

Code Camp Venue Change

Tomorrow's Seattle Code Camp will be held at DigiPen in Redmond and not Devry in Federal Way. I can now sleep an extra hour before heading over 

I'll be giving a session on buffer exploits. If you're a C/C++ developer and haven heard about them, but never really understood how they worked, you may find this interesting. It's not a "hacker training" session, but more of a practical look at how stack & heap overruns work (as well as string format bugs) and some ways to prevent them since they can be painful if exploited.

If you're not into buffer exploits, there are also other great sessions from some of the northwest's top speakers, so it's worth heading over for at least a while.

10/27/2006 10:16:47 AM (Pacific Standard Time, UTC-08:00)      Comments [1]  

Code Camp Back In Seattle!
October 28&29 (weekend days). Check out http://seattle.techevents.info/codecamp/2/default.aspx for more details.
10/4/2006 10:19:31 AM (Pacific Standard Time, UTC-08:00)      Comments [0]  

Another Neat Little Security Feature
I just found out that Outlook 2003 won't launch a URL if you click on it in an email while it's in your Junk Email folder. Assuming this is a feature and not a bug () I think that's pretty cool.
9/18/2006 7:35:02 AM (Pacific Standard Time, UTC-08:00)      Comments [1]  

What If There Really Are None Above?

Kind of cheesy, but still somewhat ironic:

9/6/2006 9:56:42 PM (Pacific Standard Time, UTC-08:00)      Comments [0]  

I'm Taking The Plunge!

Last week I decided to check in on Vista by installing build 5536 (it's pre-RC1) on a test machine. It took me around 15 minutes before I decided to upgrade my primary laptop and make the wholesale move to Vista. It's that good.

I will never look back.

9/6/2006 4:33:22 PM (Pacific Standard Time, UTC-08:00)      Comments [1]  

Atlas Deployment Performance Tip

Rohan and I were in the office this weekend working on the beta deployment of a significant Web app the team has been working. This is the first major site we've deployed that uses lots of relatively new functionality, such as clustered ASP.NET membership with custom providers, heavy reliance on SQLCLR, significant code access security configuration, and all-around complexity issues that can pop up in any place.

Overall, the deployment of the app has been incredibly smooth. The installs went without issue, the config steps were well-documented and accurate (both from MS and our internal notes), and the stars aligned to make it all work without much issue. However, we did hit a huge performance issue once we began smoke testing. Every page was taking three to five seconds to load, which was completely unacceptable.

After using some protocol analyzers and Application Center Test (my personal favorite HTTP sniffing tool) we determined that there was a 350K+ file that kept getting downloaded on each page (generated from a request for WebResource.axd). If you come across this, simply turn off debugging on the Web.config file (make sure the tag is <compilation debug="false">) so the file can be cached. After fixing this, the page response time dropped dramatically to under a second.

There are other reasons for doing this in a production deployment, but the performance is worth it alone. Scott Guthrie explains more here.

8/27/2006 2:33:42 PM (Pacific Standard Time, UTC-08:00)      Comments [2]  

Anti-virus Software Can Suck It

People who tell virus stories always seem to come across as alien abductees. Here's my turn.

I came across a random site on Thursday that was a launcher for a WMF-based virus. Although I always keep up-to-date with Windows Update, this one slipped past the goalie. All I remember was clicking from blog to blog (I don't even remember what the topic was) and I was forwarded to a page with Chinese, which launched a video in Media Player, which launched the default image viewer. IE raised popup with text box filled with little boxes (the kind you get when you're missing a language pack).

Sensing the ambush, I quickly 'Ctrl+Shift+Esc'aped (launch Task Manager) and blasted 'I, Del, Enter' (find next process starting with I, kill it, yes--I'm sure I want to kill it) four or five times (I had several IE windows up and had to nuke them all to save the world).

The threat was gone. I was safe. I slept well that night.

Late the next day, I had an urge to reboot my machine. I'm not sure why, but it just seemed therapeutic. You know what I mean.

When the machine came back up, I could no longer run non-Web .NET Framework 2.0 apps. Every time I tried to launch one, I'd get a dialog with "The application failed to initialize properly (0xc0000005)." ASP.NET 2.0 Web sites worked fine. .NET Framework 1.1 apps worked fine.

Seriously. WTF.

Rather than rebuild the machine, I decided to dig into the issue and see if I could solve the problem. The first thing, of course, was to search the Web for other people who had this issue. I came across only a single relevant post. The posted had solved the issue by taking a pass with anti-virus software.

Being a strong Microsoft supporter, I jumped at the chance to try out Windows Defender. With a name like "Defender", I figured it wouldn't be much help since it implies that it would need to be on the machine while it's being infected, which is something I haven't quite confirmed yet. Regardless, I installed it and it does have a nifty scan utility, which I ran:

Windows Defender: Master, I have uncovered many problems with your system!

Ed: Wow, Defender, that's great! What are they?

Windows Defender: You have 700 spyware threats. Would you like me to eradicate them?

Ed: Sure, go for it! You're my hero!

Windows Defender: Working...done. 700 HTTP cookies deleted!

Ed: Are you %&*#ing kidding me? I could have done that. Somehow I doubt any of those cookies caused the .NET Fx 2.0 issue.

Windows Defender: Would you like me to scan again?

Ed: No, that's okay. Why don't you take the rest of the night off? Go play with ActiveSync. He's about your speed.

(In defense of Windows Defender, it does have a nifty utility for seeing what apps are set to run at startup, currently running, etc, that also includes full names and verified publishers. However, this didn't include the malware that was screwing me, so I don't want to big up them too hard.)

Fortunately, we have a small business license for Norton Internet Security, so I decided to install it. Note that I say "decided to install it" and not "installed it" because the damn thing wouldn't take. Fortunately, the installer is eager to help. Every time it came across a "fatal error" it offered me a button so that I could try the install again. It wouldn't bother explaining to me what was failing, but at least it cared enough to want to be installed.

Up next was Panda. Ordinarily I wouldn't consider a system security company that doesn't understand the difference between "remember password" and "password reminder":

However, the guy on the Microsoft forums said it solved his issues, so I figured I'd give them a whirl for $12. Before buying the service, I decided to use the free scan, which fixes any viruses it finds, but leaves any spyware, backdoors, etc, until you pay. Fortunately for me, it managed to find and fix two viruses on the machine--all three times I ran it.

You see, the funny thing about viruses is that they spread like, well, viruses. If you don't fix them properly, they come back, sometimes immediately.

Fortunately for Panda's coffers, it also found a few backdoors on my system, so I happily paid for it to get rid of them. Everything was gone and life was good. Unfortunately, my .NET 2.0 apps still didn't work. I even went as far as to reinstall SP2 and the .NET Framework 2.0. Still no luck.

Finally, I decided to take matters into my own hands. This is the section that might be useful for other folks who run into problems AV apps don't help with.

1. I booted into safe mode. You can get into safe mode by repeatedly banging the F8 key during the early stages of the boot process while yelling profanity. You can try to leave out the yelling, although I don't know if it will work since I have never booted into safe mode without profanity. Oddly enough, the .NET 2.0 apps all worked fine, which indicated that something was funky with the stuff that got loaded in a normal boot (this is almost always the case with malware).
2. I went into my System32 directory (typically at c:\windows\system32) and sorted all the files by their dates. Sure enough, there were a handful of DLLs and EXEs that had the same timestamp, which was roughly about the time I hit the malicious site on Thursday. I deleted the files one by one. By the way, I'm kind of a cowboy when it comes to deleting system files, so you may want to exercise a little more caution. Some of the files wouldn't go, such as "obbf115.dll" and "obbf117.sys".
3. Next, I was going to search the registry to see where these files were getting loaded from. You'll typically find them in a few places like:
   1. [HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\Run] and [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run] are where Windows checks to see what command lines it should run each time you boot.
   2. [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon\Notify], which is the offender here. Each subkey points to a DLL that will get loaded by Windows for various logon events. If you're going to mess around in here, it's a good idea to back up your registry first (I don't--see cowboy note above). One effective way to tell what doesn't belong is to iterate each subkey and look at the path to the DLL. Although anything's possible, some DLLs make sense if you know what's going on when you log in, so you may not want to worry about them. However, another trick is to google each DLL to see where they've come up before. If the DLL is malicious, someone else has probably seen it already and posted it via a HijackThis log or in passing conversation. I found three subkeys that pointed to DLLs that I had just deleted due to timestamps I didn't like. I was able to delete two of them, but the third one kept re-adding itself! It mapped to the DLL I couldn't delete from before, so I knew we had a connection.
4. I opened Task Manager and killed every process it would let me kill. (Make it a game! See how many processes you can end in 10 seconds, then reboot and try to beat your best score.) After this I was able to rename the DLL (although it still wouldn't let me delete it). Fortunately, the DLL did not reappear.
5. I rebooted and returned to safe mode.
6. I was able to delete the DLL and SYS file, which is because they were not loaded since the DLL name had been changed.
7. I rebooted into normal mode and wrote this blog entry.

I know it's a long read, but I hope this helps.

8/27/2006 2:21:26 AM (Pacific Standard Time, UTC-08:00)      Comments [0]  

What A Relief!

I'm not sure why, but this made me feel validated:

I was so happy I even overlooked the grammatical nightmare.

But seriously, if your identity can't be identified, what good is it?

8/26/2006 12:20:14 AM (Pacific Standard Time, UTC-08:00)      Comments [2]