Improve Web Development

Modified Preorder Tree Traversal

2009-03-15T11:03:00.004-04:00

If you are a programmer, hopefully you've heard of this term. I learned about it 7 years ago while a big city programmer from New York looked down at a country programmer from Sussex County Delaware. He had a lot to say but of little value. When we were discussing an e-commerce platform that I had built, he asked which algorithm I was using for parent/child relationships. I explained how I had a parentCategoryID field that held the value of the parent category in the child category row. One weakness in this approach is that it is tough to find indirect relationships. Say I have top level category of animals and plants, then the next level has birds, fish, reptiles, etc. Then, the next level has the species of animals and plants. How can you create a query that says display all animals? Without multiple queries or complex queries, you can't.

lifeFormID	name	parentLifeFormID
1	Animal	NULL
2	Plant	NULL
3	Bird	1
4	Fish	1
5	Reptiles	1
6	Trees	2
7	Shrubs	2
8	Cactus	2
9	Duck	3
10	Goose	3
11	Snake	4
12	Oak Tree	6

Using the data above, I want a query that returns duck, goose and snake but not oak tree. Using this method, there is no clean way. Either multiple queries

SELECT lifeFormID, name, parentLifeFormID FROM lifeForm WHERE parentLifeFormID = 1

then...

SELECT lifeFormID, name, parentLifeFormID FROM lifeForm WHERE parentLifeFormID IN (list of ID's from the above query)

Yes, if you know that there are only three levels then you could build the query below:

SELECT lf.lifeformID, lf.name, lf.parentLifeformID FROM lifeForm lf WHERE lf.parentLifeFormID IN ( SELECT subLF.lifeFormID FROM lifeForm subLF WHERE subLF.parentLifeFormID = 1 )

But this creates code that is not very modular or efficient. A better way to perform this is using the Modified Preorder Tree Traversal algorithm.

Lets try setting up the table this way.

lifeFormID	name	parentLifeFormID	treeLeft	treeRight	treeLevel
1	Animal	NULL	1	14	1
2	Plant	NULL	15	24	1
3	Bird	1	2	7	2
4	Fish	1	8	9	2
5	Reptiles	1	10	13	2
6	Trees	2	16	19	2
7	Shrubs	2	20	21	2
8	Cactus	2	22	23	2
9	Duck	3	3	4	3
10	Goose	3	5	6	3
11	Snake	4	11	12	3
12	Oak Tree	6	17	18	3

Now lets say we want to create a SQL query that retrieves all types of animals.

SELECT lifeFormID, name, parentLifeFormID, treeLeft, treeRight, treeLevel FROM lifeForm WHERE 1 < treeLeft AND treeRight < 14

Note: 1 is the treeLeft value of the animal row and 14 is the treeRight value of the animal row

This method allows for SQL queries to be created that allow for indirect relationships to be found simpler. The one downside of this algorithm is that it is more complicated to add/edit/delete from the table. Since most applications use the view data much more often (generally around 90%), it is better to simplify and speed up the view process while losing speed on the admin side. As for the actual functions and SQL to maintain this, SitePoint has a good article with some useful code that you should be able to migrate into your application.

The History of Geocoding and Distance Calculation on the Web

2009-03-15T08:37:00.006-04:00

I have been working on a project that will allow for hunters to mark on a map where they have hunted and what animals they have brought down. The website should be launched in the next month or two. In working on this project, I've been researching what is new in the geocoding world. Not surprisingly, much has changed in the past few years. In this article, I'll be going over the old way of geocoding.

So what is geocoding?

Geocoding is the process of finding associated geographic coordinates (often expressed as latitude and longitude) from other geographic data, such as street addresses, or zip codes (postal codes). (Source: Wikipedia)

This process is vital for all sorts of applications. In the past, you purchased a zip code database from one of several vendors. I've always used Melissa Data. This would tell you the latitude and longitude coordinates on a zip code level. Census data would also be included in the database. Back then, before Google Maps and similar allowed for mapping to be used to develop applications on your website, this was about all the level of specificity you could get without spending lots of money.

Once you had this database, you could calculate the as a crow flies mileage between zip codes. This mileage can be calculated using the following algorithm from Meridian World Data.

sqrt(x * x + y * y)

where x = 69.1 * (lat2 - lat1)
and y = 53.0 * (lon2 - lon1)

Depending on the level of accuracy needed for the application, you can use different formulas. In most cases, I've found the above algorithm accurate enough. As an added bonus, the above formula is simple enough to easily use in a database SQL statement.

An example of an application would be a company that delivers widgets to a local area. Based on the mileage, the delivery charge can be assessed. Another example would be a news website that if it knows the users zip code, they can provide news that is more local.

Admittedly, the above formula won't get exact mileage for routing for a couple of reasons. One, the calculation is as a crow flies meaning that the calculated mileage takes no consideration of the roads that you would have to take. As an extreme example, imagine you are on a Delaware beach and want to get to New Jersey (I have no idea why anyone would want that). The as a crow flies mileage would be about 25 miles. The actual driving distance is closer to 167 miles. Extreme examples aside, as a crow flies mileage is usually pretty close to the actual driving distance.

The other issue with this is that you are only calculating zip code to zip code. If the origin address is in Lewes, Delaware and the destination address is in Milton, Delaware, the distance based on the above calculation would be the center of Lewes, Delaware to the center of Milton, Delaware. What if I am at the north end of Lewes and travelling to a home in the south-east end of Milton? The distance might only be 5 miles instead of the 12 miles from the center of zip code calculation.

I'm providing extreme examples to prove a point of the inaccuracy of these old methods. In reality, you can usually accept this level of accuracy. In the delivery service example, you plan to make money on some delivery (where the actual distance is less than reported) while losing money on others (where actual distance is more than reported).

Three or four years ago, this was all we had (without paying lots of money for a routing service). Good applications were still being built if not quite as accurate as everyone would have liked.

Fast forward to now and we have several providers (Google and Yahoo! among others) that provide an HTTP based geocoding service. Mapping is becoming easier to integrate into your website. The level of accuracy is at an address level compared to the zip code level of yesterday. In another post, I will discuss the current implementation of geocoding and the maturation of the applications that can be built as a result.

Graphic Design and Hosting Selection - Joomla Install Step By Step Instructions - Step 1

2009-03-03T12:18:00.009-05:00

I've been using Joomla over the past half a year now. The application works very well for professional website design projects. The Joomla application itself is free under the GPL license. I enjoy using Joomla because it allows me to focus on my clients and building great websites instead of having to program functionality that has been built many time before.

In this series of articles, I will go through the steps that I use to build a Joomla content managed website. I have been using Joomla almost exclusively for the past six months and have setup approximately 15 websites (not all launched yet). I will be going through the process that I use to setup a website.

First step, is to have a graphic designer design a great look for your website. Alternatively, you can select a template from one of several websites. Monster Templates has many great templates built specifically for Joomla. RocketTheme has also been a great source for finding a good template to for the website. While we prefer to also have a graphic designer create a custom built design, sometimes that just isn't in the budget. Purchasing a template will also free you from programming the CSS and html template files.

Once you've performed this, you'll need to select a hosting provider. I have had experience hosting websites at the company I worked for, using a local vendor and using a national vendor. My best experience has been with the national vendors. While the tech support can be a little lacking sometimes, they have invested enough time in the server setups that little needs to be asked of them. Some of the hosts that I have experience with are InMotion Hosting, Host Monster, and Host Gator. For email, I use Google Apps. It is currently free and provides excellent SPAM protection free of charge.

I would also recommend that if you are also purchasing a domain, do not purchase the domain through your host. Use a domain registrar like GoDaddy (make sure they have free DNS management). This makes it much easier to move the website if your host selected isn't satisfactory. I have some horror stories where a client's website had to go down for 5 days while the domain was transferred away from the current host. If the domain registrar was elsewhere, this would have been avoided.

After the above steps have completed, you will be able to actually create your Joomla install, install some components and modules, and begin actually creating content. I will cover how to accomplish these steps in the installment of this series.

Ubiquity: Why Catastrophes Happen

2009-02-19T22:14:00.004-05:00

An interesting book written by Mark Buchanan. The book sets out to find why catastrophes happen. What is the difference between the 1906 San Francisco earthquake to the 10,000 earthquakes that happen in California every year? Why do some wildfires cause massive damage while others go unnoticed? What about major wars compared to minor conflicts?

The book goes through all of these cases, along with many more, in an attempt to find a pattern in the difference between a catastrophe and a minor blip in time. The results, backed by many different researchers in different fields, are unsettling. In each case study, a power scale is found. Basically, an earthquake that is 1.0 on the Richter scale is 4 times more likely to occur than a 2.0 earthquake. And a 2.0 earthquake is 2.2 times more likely to occur than a 3.0 earthquake and so on. The power scale number will vary based on what is being studied.

The book goes on to talk about many more examples like wars and international conflicts. What made the World War II a huge war compared to some of the smaller conflicts that have occurred over the years? What can be done to forecast the probability of these catastrophes? Stunningly, the difference ability to forecast the severity of a war is just as impossible as attempting to forecast the severity of an earthquake or forest fire.

In the end, you learn that when something gets into a critical state, the severity of occurrences cannot be determined. A little disconcerting but also fascinating that you can look at anything in a critical state and, once you find its power law, can spot the similarities of the chance of occurrence.

The Timeless Way of Building

2009-02-10T08:07:00.003-05:00

The book, "The Timeless Way of Building", by Christopher Alexander is a philosophical book about the theory behind architecture. His main belief is that if you create the goals (or patterns) of what should be in each room then with enough thought you can create a room that meets all of these goals. These patterns, that at first seem to conflict with each other, can be put together like a puzzle with enough thought and belief that you will succeed.

As I'm not an architect, I like to take his way of thinking and try to apply it in my profession as a website programmer. Look at a website as a house. A house has rooms while a website has sections. Each room has different patterns (or requirements) which must be upheld in order to create a successful room. Each website section also has different requirements which must be met in order to create a successful website section. When you are planning your house or website and writing down all the patterns that must be met, some of the patterns will seem to conflict. The authors main premise is that with enough creativity and belief in you ability to find a solution, you will be able to meet all of these patterns to make a great room (or website section). Then, each room or website section will have to be pieced together to create that great finalized home or website.

While we've always kind of inherently known this to be true, the way he describes it puts words into the general feeling we've all had when building these websites. While the scope of the book is completely about building houses, communities, or towns, you will be able to transfer its meaning to your area of work.

As an added bonus, you may also get inspired to design your own home (architects beware!). When I have the ability to build a new home, I will now certainly create the general blueprints using the book and the two other in the series as a guide. I've already got an outline in my head about all of the rooms that I'd like and the general feel that I would want to go for in my home. Obviously, I'll hand these rough draft to a professional to validate that plans before actually building the home.

This book should apply well to any profession, including web development and programming. The idea that with enough planning and belief in finding a solution, you can create a solution that meets all your goals without eliminating any. At first, this sounds too pure and simplistic to actually work. While not easy, it certainly is able to be accomplished if you believe it can be and are willing to put in the work to make it so.

Selling A Product Online

2009-01-31T09:38:00.005-05:00

I get asked often from friends or clients about e-commerce. Most people want to get involved with e-commerce in some way because everyone senses the easiness to make a profit. I'm not here today to discuss the validity of how easy it is to sell online. What I am going to do is go through the different options out there.

You can go with one of the big guys, like EBay, Yahoo! Store or Amazon WebStore. These options give you a certain amount of credibility and traffic. As you might expect the take a cut of the profit through a monthly fee or a percentage of sales or both. In return for this they take care of the merchant account and a certain amount of the marketing. If you are in the right market, these options can work.

Another option is some of the middle-tier SAAS e-commerce sites out there. Some of the prominent ones are Shopify and Vcart. These services will generally cost less (when considering per sale fee) then EBay, Yahoo! Store or Amazon WebStore. While the cost is lower, the marketing that the bigger guys help with either goes away entirely or is greatly reduced. These can be useful if you want to do a lot of marketing for website but don't have the technical expertise (or desire) to manage a custom built store.

The final option out there is to use an open-source commerce solution. You can go two ways here. A pure commerce solution like Miva Merchant, osCommerce, Zen Cart, and Ubercart. Or, you can use an extension of an open source CMS solution out there like Joomla or Drupal. While have experience with most of these options here, I prefer to go with a pure commerce solution because that is there main focus while the CMS extension is just an afterthought.

I will go more in depth with discussion about some of these options as I use them. Currently I am developing under an osCommerce platform, Joomla and Drupal extensions. While osCommerce is a little rough around the edges, if you put the time in, it seems to be the best of the ones I've tried. I still intend to try out Miva Merchant, Zen Cart and Ubercart so I can have a broader view of the options out there. As I try these out I'll have some more posts here.

Free CMS Websites?

2009-01-27T07:33:00.005-05:00

So at Fine Line IT, we mainly develop websites using the Joomla platform. We have had great success using the platform so far with the few expected annoyances. For the most part it is free, enough of the components / modules work great and the platform is easy enough to customize for your needs when the Joomla component doesn't work just right.

The main reason anyone goes to an Open Source CMS platform is that it is free. Nothing beats free, right? When, we began the process of starting Fine Line IT, we decided we like free. We looked intensely for all products that were of no monetary costs. While other options exist in the free CMS arena, for various reasons, we chose Joomla.

So far, we have been impressed with Joomla. We have been able to accomplish everthing we want in a website using the CMS application and several of its components. I will begin to discuss the different components we use on another post. Suffice to say, if you look hard enough at the 1,000's of components Joomla developers offer, you can find a good quality free one that is close enough to solving your solution (forms, photo galleries, google map, etc) that it can be customized slightly to be perfect.

In the coming months, I will descuss in detail some of the components that we use and how we use them to build great quality websites.

Reopening of Improve Development

2009-01-26T12:51:00.003-05:00

The improvedevelopment.com blog has gone live today. This blog's intended audience is for like-minded programmers and web developers. The blog will discuss anything in the programming / web development world. This could be anything from discussing the use of an extension in Joomla to talking about project management of web sites.