The life of a code monkey

Learning – Understanding – Exploiting

 I have always enjoyed figuring out how things work, specifically computers and technology. Learning about how something works can give you great appreciation for the work that went into creating it, as well as insight into how to exploit it.

Countless times I have seen web sites attempt to limit a user’s interaction without requiring authentication. Examples of this would include polling web sites, voting web sites, or as in today’s example, web sites that limit the number of articles you can view.

It is nearly impossible for a web site to conditionally limit your access, without requiring you to authenticate.

 A practical example

 The web site, www.spokanejournal.com is a small independent business newspaper. If you navigate to the web site, you can view various articles. You will be able to read the first three articles that you click on. After that, however, you will get only the first few lines of the article followed by a prompt asking you to subscribe or purchase the article. In fact, if you stay on the same article and hit the refresh button three times you will get the same behavior.

So what is going on?

 Learning

 I would like to re-iterate my point from above, it is nearly impossible for a web site to limit the number of articles without requiring you to login.

When you go to a web site, your computer sends a request to some other computer on the internet to get the web page. To use a simple analogy, this is like you sending a letter (back when were written on paper and sent through the postal service) to a company asking for a brochure. When the company receives your letter they mail you back the brochure. Bringing it back to our example web site, let’s assume that the company has created a rule that every person is only allowed three free brochures. Let’s consider how to implement such a rule.

The first logical choice is to simply look at the return address on the letter, and only send three brochures to that address. This works, however it makes the assumption that there is only one person living at each address. By doing this, you have limited everyone living at the address to only getting three brochures. This could be a pretty big problem if there are a lot of people living at the address. In computer terms, this is known as IP address filtering (and a good discussion for another time). You could think of a large office, or a college dorm as examples where many computers (people) could have the same internet address (living at the same address). The point of this, is to understand that in the world where you communicate simply by sending letters back and forth, it is very difficult to identify exactly how many brochures we have sent to any one person.

Since we cannot identify a particular person just from the return address information, what if we try something a bit sneakier? As the company, when we get a request, we send back the brochure and a preaddressed stamped envelope with a unique number on it. Then if we get another brochure request in an envelope with a unique number on it we can easily look up how many brochures we have sent to that person. This assumes that people will reply back with the preaddressed stamped envelope rather than getting a new one. This seems a little silly to make that assumption, but this is how many web sites work.

 Understanding

 The technical term for storing information (the unique id on the envelope) is a cookie. This cookie is not limited to just a number; it can be anything we want to scribble on the envelope. When you open to a web site, it can store cookies on your computer. Then when you visit the web site again, it will send the cookie’s information back to the web site.

It is worth noting that not all cookies are bad or malicious in their intent. For more information, see here, and here. Because they are an easy way for web sites to keep track of a user, they are used by nearly all web sites for one purpose or another.

Back to our example web site. Since we have not logged in, and the web site seems to keep track of how many articles we have viewed, it is likely tracking us using a cookie. To test our theory, clear the cookies on your browser and then go back to www.spokanejournal.com. We can now view another three articles. Theory confirmed.

 Exploiting

 Clearing all of the cookies every time is not a clean solution. It will wipe out cookies for other sites that we may want to keep. For the next part, we will be making use of Chrome and one of its extensions EditThisCookie.

After installing the extension, let’s clear our cookies and go back to www.spokanejournal.com. Using EditThisCookie we can see that the web site adds the following cookies, __gads, _ga, _gat, PHPSESSID, and shopper_id.

Now simply open an article and hit refresh until you are no longer able to view the entire article. From here simply delete a single cookie and hit refresh until we determine which cookie(s) are used to track how many articles you have viewed. After a little trial and error, we can determine that the shopper_id cookie is the one that is being used to track the number of articles we have viewed.

Using the block cookie button, we can block this cookie from every being created. This will prevent the cookie from being created. To return to the previous analogy, this is the equivalent of erasing the information that was written on the envelope.

Clearly anything worth doing is worth over-doing. You can see my previous two attempts here, and here. A co-worker pointed out that my previous attempt did not handle the case where there was a word after an acronym, ie. EndPointIVRKey. Religious wars about acronyms used in CamcelCase aside, it sounded like a fun regex to try and write. The following is the end result:

static void Main( string[] args ) { var tests = new Dictionary<string, string> { { "EndPointWeb", "End Point Web" }, { "EndPointIVR", "End Point IVR" }, { "End_Point_Web", "End Point Web" }, { "End_Point_IVR", "End Point IVR" }, { "end_point_ivr", "end point ivr" }, { "EndPointIVRKey", "End Point IVR Key" }, { "EndPoint_IVRKey", "End Point IVR Key" }, { "End_Point_IVR_Key", "End Point IVR Key" }, { "End_pointIVR_key", "End point IVR key" } }; var regex = new Regex( "((?<Letter>[a-z])_?(?=[A-Z]))|((?<Letter>[A-Z]{2,})_?(?=[A-Z][a-z]))|((?<Letter>[a-zA-Z])_+)" ); var sb = new StringBuilder( foreach (var test in tests) { string result = regex.Replace(test.Value, "${Letter} "); Console.WriteLine( "{0} -> {1} : {2}", test.Key, result, result == test.Value ? "PASSED" : "FAILED" ); } Console.ReadLine(); }

Output:

EndPointWeb -> End Point Web : PASSED

EndPointIVR -> End Point IVR : PASSED

End_Point_Web -> End Point Web : PASSED

End_Point_IVR -> End Point IVR : PASSED

end_point_ivr -> end point ivr : PASSED

EndPointIVRKey -> End Point IVR Key : PASSED

EndPoint_IVRKey -> End Point IVR Key : PASSED

End_Point_IVR_Key -> End Point IVR Key : PASSED

*NOTE* There is an update to this post here.

A while back I posted a small regex to properly space CamelCase text so that it can be displayed in a more human readable form. Recently I discovered a few issues with it. 

It does not handle all CAPs at the end of a string

It does not handle underscores (because why not making the naming scheme as difficult as possible and mix CamelCase and snake_case).

Here is my modified solution as well as a few test cases.

static void Main(string[] args) { var testStrings = new[] { "EndPoint_Web", "EndPoint_IVR" }; var regex = new Regex("(?<=[a-z])(?<Ignored>_)*(?<Letter>[A-Z]+)(?=[a-z]|$)"); foreach(string str in testStrings) Console.WriteLine("{0} -> {1}", str, regex.Replace(str, " ${Letter}")); Console.ReadLine(); }

Output:

EndPoint_Web -> End Point Web

Hi Gary,

Unfortunately the answer is not likely. RS232 ports are just simple ports that can be used to send and receive 1s and 0s. Each individual device interprets these signals in their own way. your best bet is to look into information from the manufacturer of the projector lift to see what their device supports. 

A friend of mine was working in WPF and made the comment that it would be great if there were a IMultiValueConverter that could take in an arbitrary expression and evaluate it. It sounded like something that would be fun to create, so I did. Code available at https://github.com/Keboo/Binder

Background The initial desire was for the converter to be able to take an expression, say "!({0} || {1}) && {2}", where each of the string format parameters would represent a value coming from one of the bindings. To do this, would either mean parsing the string myself (yuck) or passing it off to the framework to dynamically convert it into something I could easily consume.

On a recent project I came across some code that gave me a good disbelieving laugh.

On one of my projects I had the need to be able to send generic events to various objects. There is plenty of information on this type of observer pattern, what follows is my implementation. 

I have often taken advantage of the Insert and Add methods on the List class. Quite often I have used the insert method to place an item at the beginning of a list. What I had previously failed to consider was the performance difference between these two methods.

Consider the following code:

const int NUMBER_OF_ITEMS = 100000; var sw = Stopwatch.StartNew(); var listA = new List<int>(); for (int i = 0; i < NUMBER_OF_ITEMS; i++) listA.Add(i); sw.Stop(); Console.WriteLine("Add: {0}", sw.Elapsed); sw.Restart(); var listB = new List<int>(); for (int i = 0; i < NUMBER_OF_ITEMS; i++) listB.Insert(0, i); sw.Stop(); Console.WriteLine("Insert: {0}", sw.Elapsed); sw.Restart(); var listC = new List<int>(NUMBER_OF_ITEMS); for (int i = 0; i < NUMBER_OF_ITEMS; i++) listC.Add(i); sw.Stop(); Console.WriteLine("Add (no resize): {0}", sw.Elapsed); sw.Restart(); var listD = new List<int>(NUMBER_OF_ITEMS); for(int i = 0; i < NUMBER_OF_ITEMS; i++) listD.Insert(0, i); sw.Stop(); Console.WriteLine("Insert (no resize): {0}", sw.Elapsed);

My initial thought was that Insert would be slower, but the difference was much larger than I had expected. On my laptop I got the following results:

Add: 00:00:00.0016292 Insert: 00:00:03.8859294 Add (no resize): 00:00:00.0011914 Insert (no resize): 00:00:03.8573192

From the documentation:

Add: If Count is less than Capacity, this method is an O(1) operation. If the capacity needs to be increased to accommodate the new element, this method becomes an O(n) operation, where n is Count.

Hi,

I assume you are referring my garage door opener that I built. The program that I wrote was done in C# using VS 2010 express. The board that I purchased has code examples in VB6 as well it should work just fine (http://www.controlanything.com/Relay/Device/R220HPRS). If you have any more questions let me know.

This is an argument which I have seen hashed and rehashed by many developers. Should you consume existing code or write everything yourself - in other words, reinvent the wheel.

Rather than rehashing the entire argument, here are the general argument points:

Rewrite code (reinvent the wheel)

Removes external dependencies

Forces you to know and understand the code

Consume existing code

Less time to implement  and start testing

Takes advantage of other’s knowledge and experience

Quite often there are good reasons to rewrite functionality that already exists. Many advocates of re-writing code will make the argument that it promotes innovation and pushes technology forward.  When you have written the code, it should work exactly as designed and you should know what every line of code is doing.  It is good to innovate. It is good to want to create something better than the current status quo. To those that are striving for that, I applaud you.

There is, however, a flip side to the coin; an argument which the advocates of code reuse will continuously cite. This argument suggests that people who reinvent a piece of code do so in a manner that is significantly inferior (typically in the form of bugs, performance, completeness, and conformance to standards) to previously written alternatives.

"Less bugs" is left out of the lists of pros for each school of thought, because both sides will claim that their option results in fewer bugs. The advocates of rewriting code will argue that third party utilities are often not bug free; after all what code is ever 100% bug free? The advocates for consuming existing code will argue that rewritten code is more likely to have bugs.

I think people from both schools of thought are correct to a certain degree, and there can be wisdom which may be gleaned from each. Blindly adding code to your code base which was posted on blogs, stack overflow, newly created open source projects, open source projects with little to no documentation, and vendors without a proven track record is asking for problems. In these cases, you are probably better off writing your own code, or at least investing the time to really understand and fix existing code.

Before adding an additional dependency to your project consider the following:

Do you know how to use the code? If not, you may spend a very large chunk of time learning how to use it properly. Using someone else’s code incorrectly is just as likely to cause bugs as writing your own code. Yet you must also consider the tradeoff, the time it takes to learn a library is time that could be spent coding your own.

Does the code provider have a good track record? Using well established projects which have been around for a while, are well tested and documented, and have good community reviews carry very little risk. Always consider the source of the code. If the code is open source, consider using it as an example on which to base your own code.

Now to switch to the other school of thought. Reinventing functionality when superior alternatives exist is just as bad as consuming poor code. This is especially true when writing code for protocols and specifications that have a RFC. In these cases it is best to really research alternatives before considering writing your own code. Most high level languages today have a wealth of functionality built into their frameworks. Do not re-implement this functionality. If you find yourself wondering if something is already implemented; stop coding and start Googling (or binging for those die-hard Microsoft fans). In almost every case your framework’s implementation will be faster and less buggy.

Let’s use the .NET framework as a specific example. Microsoft has built a very powerful library on which to base your software; take advantage of it. The more you familiarize yourself with the wealth of the framework the less code you will find yourself needing to write. As a result, you will end with much better code.

With .NET 3.5 and above, we have the ability to add functionality to objects which we are not able to modify directly. As the name suggests, it is used to extend the functionality of an object. Too often, I find extension methods whose purpose is to not extend functionality. Consider the following examples of poor choices for extension methods:

public static int Add(this int first, int second)

This is simply re-implementing the plus operator in a much stricter context since it only accepts ints. Do not use extension methods to re-implement operators. Your new way of using an operator is not better than the one built into the language.

public static IEnumerable<T> NotNull<T>(this IEnumerable<T> enumerable)

This method would do the equivalent of Where(x => x != null), so do not wrap existing functionality just to get a different name. The development who has to maintain your code will appreciate it.

public static void ConvertAllAndEmail<T>(this IEnumerable<T> enumerable)

This method is doing way too much. The functionality it claims to provide does not seem to relate to IEnumerable<T>. Do not create extension methods on types simply because you can. Extension methods are meant to extend functionality; to make an object richer and easier to work with. Think carefully before you add an extension method to System.Object.

There are plenty of other examples, but to be brief- learn your language/framework. Use the functionality already provided to you when possible, and only when necessary create your own custom implementations. As developers we should strive to work smarter not harder.

In conclusion, before you go trying to re-invent the wheel consider these questions.

Why do you need a custom implementation? If you can’t write down an answer to this question in a couple sentences, then the problem is not understood well enough to design a solution.

Does it make the code more readable? Readability is typically a judgment call (and the source of many debates between developers). Typically shorter simpler code is best. Would a new developer be able to read the code and understand what it is going on?

Are there solutions that have already solved the problem? Typically you are not the first developer to want something. Search your platform/framework to see if a solution has already been provided. Quality third party solutions may also be considered if your project permits. If you cannot find any existing solution, consider the possibility you are doing it wrong.

These references should not be considered scholarly works, rather merely examples of the debate between code consumption vs. re-inventing the wheel.

Reverences:

[1]:  Reinventing the wheel vs. code reuse

http://www.lotushints.com/2010/05/reinventing-the-wheel-vs-code-reuse/

[2]: The “Reinventing the Wheel” Anti-Pattern

http://www.selikoff.net/2009/12/10/why-reinvent-the-wheel/

[3]: Re-inventing the wheel – Is it bad? (Stack Overflow)

I have recently become a huge fan of SkyDrive.

By default the SkyDrive directory is located C:\Users<USER NAME>\SkyDrive

I am lazy and having a drive letter mapped to this directory made scripting and command line work much easier.

Using the SUBST command you can map any directory to a drive letter.

The syntax is: SUBST :

Assume that you are worked with a flagged enum value. Perhaps something like this:

[Flags] enum FinishTypes { None = 0, Gold = 1, Silver = 2, Bronse = 4, Medals = Gold | Silver | Bronse, FirstRunnerUp = 8, SecondRunnerUp = 16, FirstLosers = FirstRunnerUp | SecondRunnerUp, All = Medals | FirstLosers }

We want to write a method that returns a boolean indicating if the passed value is made up of a single value, or multiple items from the enum.  In our case, we would expect true to be returned for FinishTypes values of Medals, FirstLosers, and All or any bitwise combination of values.

private static bool IsComposite(FinishTypes finisherType) { //code? }

Before looking at one possible solution, give it a shot. There are multiple (correct) ways to solve this.