At the end of this semester, I am gearing up for what will be a drastic change in my life.

My plan, in short, is to sell my car when it gets back from the body shop in late March/April. At the same time, I will be putting together a unique portfolio demonstrating my abilities and worth to a potential employer.

When I do finally sell my car, I will use the money to help me relocate to New York City, NY.  My projection leaves me looking for a month or greater sublet in May, as long as the car sells by then.

I will sublet while I look for a php position in the city. Once obtained, I’ll then look for residency depending on pay.

So I will have a lot on my plate these last few months in South West GA, if you have any resources in NYC and want to help me make the transition easily, all help is welcomed, and I will do what I can in return, if there is something I can do.

In the past month, I was involved in a ‘hit and run’ by a meth head who railed me in the back doing about 50 mph.

I also moved, into the middle of uptown Albany. I am still moving things around trying to get everything set back up.

Last but not least, my cars engine blew. It is being rebuilt right now.

It doesn’t help having a full time job and being a student full time either.

I did go to mardi gras in New Orleans, though. =)

I’ll write more articles hopefully when I have some time on my hands.

I will say this a lot but I really mean this guys, preparation and planning really helps. Before coding anything, sit down with a pencil and paper or your favorite UML editor .

Lets assume you already have a database set up for the required user input. I would go into schema design, but that is a different topic. So you want to take user input, process it and store it in the corresponding database table.

If you have your database set up correctly, this is actually not that daunting of a task. Often, developers attempt to check the submitted data to make sure it conforms to what THEY want (size, format, integers, html etc). This is the wrong approach, we should be organizing and formatting our data to conform to what the DATABASE (or model) wants.

A huge issue in application security is data integrity. You can manipulate user input all you want, but when you pass the data to the database, if something happens during storage that alters the record (injections, incorrect format, anomalies) then you now have arbitrary data in your application flow that can cause all sorts of problems down the road. We need to be in control of our data and know exactly what it looks like wherever it is in our application.

What you should do first is make a list of the variables that your form will handle, that will be inserted into the database (give yourself room if on paper). Underneath (or wherever) each variable, put the data-type and flags set for it in the database. {varchar(50) primary key}.

Beside each variable you will write your rules. Here you will write rules to insure the data complies with the structural format you expect in your application. I find it easy to record these as questions (is it an email address? is it a phone number? is it xml?).

After those rules, we need to add a few more. Write down your database rule requirements at the end of your previous list. (eg. is it unique?{primary key} is it less than 50 chars long {varchar[50]} is it an integer {int}). We write these rules last, as these will be the last ones processed before being sent to the database.

People may wonder why we have a global set of rules. Shouldn’t the model handle database rule logic, controllers handle application logic, etc? This is one way to do it, but I prefer having one, big, complete rule set for our variables.

The reason is not only integrity but usability. You NEED client side verification while doing form processing. If you have a multi-part form, then you need to be doing verification at each step. People get extremely frustrated with filling out forms, and even more so when they have to do it multiple times. So why should we ‘OK’ form data on the client side only to have it fail database logic verification? Users are going to become untrustworthy and feel as there is something wrong, and question the safety of their data.

If we have uniform rules that are used to check the data, then if it passes client side verification, then it should pass both application and database logic verification. In zend framework, we have decorators and highly extensible builders that make this task easy. If you are not using ZF, take some of the following steps.

Use your brain and come up with a modular solution for classing your data verifications. This should be extensible in whatever way you seem fit($username = new form_db_users_username($_POST['uname']); ) ($uname = \Form\Db\Users\username($_POST['uname']);)

As I have indicated in previous posts, I support proper OOP usage, and believe highly that it can solve many poor programming habits. Find a good OOP approach for your form creation, or use commonly used programmatic designs if using a framework.

A good example of this, is having an extensible Form/Database interaction class that is meant to be extended by either a table or property class. For instance, we have a ‘username’ class extending our database class which is an extension of our abstract Forms class. The ‘Forms’ class is meant to be extended by a class determining the form’s function.

Even better we can use polymorphism, late binding or any sort of overloading to make a template for our extended classes to follow. For example, must-have methods such as (isValid() and xmlRules()). Where isValid() would contain logic to step through each rule and validate the given variable. xmlRules() could return our ruleset in xml format which could be loaded and parsed by a javascript function on the client side. Even more, you can add annotations to your class file that show what rules the data should pass along with how it correlates with the database (data-type etc).

Hell, why not think out of the box. Store your rules in xml or in the object in another easily parse-able format and have your “isValid” function parse the stored rules and use them for validation.

If you build the functionality correctly into your application, this will give you a centralized location for data verification. If you use the above method, than any problems with validating a rule can easily be debugged and modified using a single rule set which is then used throughout your program.

Using these practices will keep you from random logic errors stemming from arbitrary data that may have been mangled during your application flow or storage.

Stay tuned for more tips…

For now, lets talk about human verification.

I am an opponent of CAPTCHA based verification mechanisms. By the time you make them un-parse-able  by OCR engines they have become illegible, and makes legitimate use of your applications much more difficult. I am a proponent of easy logic questioning. Just make the layout polymorphic and un-parse-able.

The first thing you need is question templates. If we were doing mathematical logic, some examples of templates could be:

‘What is the sum of ? and ?’

‘If you add ? together with ? , what is the result’

‘? is subtracted from ? , the answer is’

Then you would have a multidimensional array with the key identifying the variable used in the logic question, in this case numbers. The dimensions will have different levels of obfuscation, such as medium and hard, with the original array index being the starting number.In this case, I will only have 1 alternate level of obfucation, with the worded number being the alt, for example:

$logicVariables[1] = ‘one’;

// This is just an example of the MD array, I wouldn’t use this sort of obfuscation as it is confusing

$logicVariables[1][medium] = ‘one’;

$logicVariables[1][hard] = ‘won’;

Now you will need a scrambler function that can scramble your template and variable a bit, to make it harder to parse. For instance running one of our sample templates could come out like this:

‘What is the sum of ? and ?’

// scrambled ex 1 , hard scramble

‘Wh@t. is the sum; of “?” and /?/’

// scrambled ex 2, soft scramble

‘What is the sum  of ‘? and  ?’

This function can make passes on the template and randomly change the character it lands on to an alternate obfuscated version like above, or if a space, could add another space or arbitrary characters to try and trip up parsing engines. It can change delimiters around the variable placeholder, change capitalization for poorly coded engines to fail, etc.

Make this all into an extensible class. Make it start by grabbing a random template out of your template database (hardcoded, sql, flat config file etc) and choose random variables (same way), then calculating the answer to save in the session (not cookie). Then run the variables and template through your scrambling function which uses a random variable that determines what setting it should be scrambled at, and how many passes to make. At the end, assemble the question and display it to the user.

You should add templates and variables(if needed) to your database often, and should often run QA tests, as well as keep a reporting function to log incorrect attempts, complete with what the question looked like, what it was before scrambling, the correct answer, and the answer given by the user (maybe even allow comments).

Guys (and ladies) this isn’t a hard approach and it would solve all this captcha bs. If they cant break your CAPTCHA with OCR, they will outsource it to Nigerians(jk) (…but yeah, they seriously will)  and you cannot stop that (maybe a time limit or something, but we fall back to usability, if I knew more about outsourcing this stuff, I could design you something).

I will add an example class later that can give you an idea on how this all comes together if you are having a problem understanding or visualizing it.

For one thing, drop the ending delimeter ‘?>’ from your include files. This is an old trick, but it really is important. It makes sure that your output stays untarnished, which is expecially important if you are doing output buffering or dealing with sessions. Any new lines and white space after the ‘?>’ is automatically sent to output. This whitespace could be added from your editor accidentally when it is saved or transfered, so this is a very helpful tip.

Another thing is in your class files, you should be using correct OOP. Google design patterns and identifiers. Make sure your private methods are private, protected methods are protected and final properties are final. Do not get ‘public’ crazy everywhere. Also static methods and properties are essential to learn if you are unaware, since we now have late static binding in version 5.3.

I know one of the great things about php is that it does not care about data types, however we should know what our data is, and what format it should be in. We have plenty of type-hinting and casting in PHP, so now is a time to use them. I even hear we are going to be getting return type hinting soon, so there is no reason for you not to become aquainted with it. This also helps us in debugging and maintaining as we know what each variable is intended to be. If you are unsure of type-hinting, please take some time to look it up and understand what it is.

Even if you are not using an MVC architecture, we should still be separating our view logic from everything else. I am against the masses when I say this but it is perfectly fine to have PHP and HTML together in the same script, but you should do so neatly, and try not to echo any kind of html from php. You should setup the elements in html and plug in the dynamic data with a simple line of php.

Go through some tutorials for ‘C’ and ‘C++’ programming, and use the ideas and structure to help your php applications conform to a more clean and optimized scale than what you already were doing.

Listen, SQL optimization and manipulation is it’s own role. Don’t think just because you know how to C/R/U/D that you know sql. There are so many things to learn that can help your application and speed it up ten fold.

I am not going to give a lecture on proper SQL, simply because I am not qualified. I have studied for the MYSQL developer exam, but have yet to take it.

I will, however, suggest a few things to look up if you are unfamiliar with them. Aliasing is a pretty powerful method for organizing your queries. You can alias a table and use the alias as the identifier later on in the query to shorten your overall length and keep it much more manageable.

When selecting records from a database, STOP using the ‘*’ character. You should be selecting only the columns you need. If you need them all, then it is up to you, but I always define the columns I need, even in that case. This keeps structural lookups from happening.

Be aware of special database functions while inserting/retrieving/updating records. You can modify the data on the database so that it is properly formatted for insertion or retrieving. You can also count records, etc. as your databases indexing system is undoubtly faster than selecting everysingle row and counting the resultset.

The most important query improvement I can suggest to developers are joins. Joins allow you to select columns out of one table, and add them to the results from another table. This solves the problem above for multiple queries for member group selection. For example, look at the following query.

“SELECT * FROM members JOIN groups ON members.gid = groups.id WHERE members.uname = ‘joey’;”

This may not be 100% correct as I don’t have access to a db for testing, but you get the picture.

So for this installment, read some advanced SQL tutorials and remember that the database is a very powerful aspect of your application and any sort of interaction with it should be optimized.

Look over the concepts of security and design in a few tutorials I created a while ago on a site that a friend of mine runs, dream-in-code. This is a great resource for programmers of any language. Also look at the “52 weeks of code” challenge they are having here .

PHP Security Crash Course by joeyadms

Professional Level Login Design Pt. 1  by joeyadms

Professional Level Login Design Pt. 2 by joeyadms

I have several other tutorials and code examples on D.I.C that can be found by clicking here

This is improper error management. In order to analyze the performance, security, and usability of our applications, we need to know everything about each request and response as possible. This is especially true for errors.

First, if you are using vhosts, you should have special error/access logs for each site stored in that sites main directory (outside wwwroot). If you are unsure how to do this, check your web server’s documentation. For example, on my development server I have each websites main folder in the user ‘www’ home directory.

For example, a site I manage, called lousvoice.com, is located at ‘/home/www/lousvoice’. Inside that folder are 3 directories, live/staging/dev (if you read yesterdays tip you will have an idea what they mean). Inside those directories are more directories for application configurations, libraries, and a ‘logs’ directory. There is also a ‘public’ director which is mapped to the wwwroot of that version of the site. This keeps all of our important data behind wwwroot.

This is a sample vhost file (called lousvoice.conf) that contains the special directives for creating individual access/error logs in those directory. Using this, you can already imagine how much easier it is to parse and manage individual sites you are working on.

Now lets get to software error management. Bottom line, you should have an easily usable and extensible error and log classes. Note that i said error AND log classes. They have two completely different objectives.

The error class should contain static error codes/messages. This will keep all of your error output easily grep/searchable and be informative. It should allow extension, and with each extended class, should add more definitions to the error codes. For instance, our global error class might contain error codes for pages that are not found etc, while our database error class might add more definitions pertaining specifically to our database (such as connectivity issues, database not found etc). Then in any area of your application, you can call the closest error class (closest class extending our ‘master’ error class) and send it the error code plus any information pertaining specifically to what happened (more info the better).

The error class should parse the error passed to it  and depending on how serious, be silent or take action in redirecting or informing the user what has happened. UNDER NO CIRCUMSTANCE SHOULD YOU EVER DISPLAY RAW ERROR DATA TO THE USER. This is not only a huuuge security risk, it is very unprofessional, usually uninformative to the user and will cause confusion. Fail Safely, and with good information explaining what happened.  Either way, the formatted error message (eg. ‘Fatal 700 : Could Not Connect To Database : host/user/pw’) should be sent to our ‘logging’ class to be logged.

Our logging class will setup information such as where the logs should be saved (disk, database, etc) and how to save it (such as maximum log size, archiving etc). It should then have a method for logging the pre-formatted error string sent to it. It is a great idea to make your logging class extensible, in case you want to store specific logs in different places. For instance, I may have my database log manager set the log file name to ‘db.error.log’ which is distinguishable from our global error log that apache uses (‘httpd.error.log’ as seen above) and our applications normal log file (perhaps ‘app.error.log’). These naming conventions will make it very easy, using some command line kung-foo , to examine your applications performance etc.

There is not much more to say on the subject. I will post some classes later, including some special ideas using late static binding in php 5.3 to get your mind going. Log and error management is a must in any production environment. Without it, you are blind as  to how your application is performing and can be extremely difficult to determine such things like security breaches and data integrity.

Stay tuned for day 4, and error/log class code examples.

Whether you are a single at home developer, or a member of a complex development team, you can benefit from version control systems. I will not go through explaining the intricacies of each system (check the footnotes for links), but in case you are unfamiliar with version control, I will give an overview.

Version control systems are highly collaborative storage mechanisms, called repositories (or repo),  for storing application code.  The current working code that is being developed is located in a central location. Each developer then pulls a copy from the repo, and works on it on  his/her own machine locally. When finished working (and testing) the modified code is updated in the repo.

So what is so special about that? Nothing, actually in nuts and bolts, a repo is nothing more than an archiving system.  The real power of version control comes with the tools integrated.

Keeping the working code in a central managed location is the heart of version control systems (VCS). This is a fantastic idea, as you know everyone has an up-to-date version of the application. There are built-in conflict resolution tools in most VCS as well.  So if you are trying to update some code in the repo, but your colleague has made a change to it since you have grabbed a copy, you can analyze the changes, checkout the most recent version, continue testing, then update the code when you are finished. Some systems allow file locking, which is  an option that only allows 1 developer at a time to work on a specific file. If you have a highly modular layout, this is a great option, however on bigger teams with larger files, this may be too much of an overhead. Mastering conflict resolution flows for your VCS is the most preffered way for synchronous and seamless collaborative development.

Perhaps the best part of version control is revisions. Everytime the code in the repo is updated from a developer, it marks the changes and a new revision  is created.  This means if we find a problem in our current revision, we can always roll back to the previous working copy. This also allows us to track changes, so debugging problems between revisions is trivial. Say in revision 3 we encounter an error that could not be recreated in revision 2. All we have to do is check the  logs on the version control software to find out what changes were made. This can assist us in debugging and analyzing our application.

On top of this, most versioning software has access lists. This is one  of the biggest selling points, imho, for larger corporations. Leaking private confidential data, such as application code or binaries, is a serious problem. Being able to tell who has what copy and made what changes at what time will assist in cracking down on any outsider sharing of data, as well as malicious insertion/modification of program code. As a note, version control is also used outside software development, for keeping control and access over confidential data and schematics.

Branching and tagging is another huge plus of using version control. Say you are working on an application, and have finished a stage of development ready for release, but of course,  you still want to keep working on the application. You take your tested working copy in the repo, and then copy it into a ‘branch’. You can give this a descriptive name, such as ‘release1′. Now people can grab the released version of your software, while you keep working on the latest version. At a point in time, you may want to fix something you find wrong in release1. All you have to do is grab a copy of ”release1′ from the repo, make your changes, then update the branch in the repo.

Tagging is basically, and sometimes literally, the  same thing as branching. The difference, is tagging is often used to identify code revisions in the past that are unmanaged.

If you are familiar with linux distributions you may be having dejavu. Does current and stable ring a bell. Think about it. Current is the code in progress at the time and stable  is a branch for code that has been tested and deemed ok for common use. There may be tags, or branches, of older versions of the software as well.

Now last but not least, my favorite reason for version control. Security. . Let me say this in caps so everyone gets it. STOP MAKING DIRECT CHANGES TO YOUR PRODUCTION WEBSITE/SOFTWARE. We are all human, we make grammatical and syntax errors all the time and have to clean, re factor and debug our way out of them. Using version control, it is possible to never allow a developer to modify source on a production system. Here is my, single at home developer, way of doing things using version control.

We have a production server which is web facing that serves our application. There is also a repository (or repositories) for holding our application code. Our repo has our working version(trunk), a “staging” branch and a “live” branch. We have the developer network which is confined, and has no access to the production network systems.

We have a development server which is a copy (hardware/software) on the developer network, which has a user for each of our developers as well as a corresponding vhost. Our developer workstations are connected to this network and securely map their own home folder on the development server. A developer checks out the latest copy of code from the repo, and makes changes and modifications. He views his changes and does testing through his vhost on the development server. Whenever he is satisfied, he simply updates the repo with his modified code.

Once the team is confident with the current state of the trunk, they copy the current version into the “staging” branch. Depending on topology and the environment in which we are developing, there is a vhost for the application on either the development server (for project managers) or a vhost on a web facing system (for clients) that requires authentication and in some instances, as with project managers, only access is granted to certain machines on the network. This vhost is for our staging branch. The code is checked out into the proper location, and the vhost points to the wwwroot of our application. While we wait on our client or manager to approve/deny the revision, we can still be working on the current version (trunk) of our software.

If the code needs to be edited, the developers can simply grab the branch from the repo, make changes, then update the repo, etc..

Once the revision is accepted, then we can copy our ’staging’ branch into our ‘live’ branch. Then a systems administrator in control of the production server simply grabs the newest copy of the ‘live’ version which updates our application on the web. Never did a  developer have to touch anything on the production server. If we need to quickly fix an issue, we simply grab a copy of the live branch, make changes, then seek approval if needed, update the live branch, then have our administrator check out the newest version of the live branch (this will  only update changed files, not replace every single file, which is a plus for performance).

Another benefit to this form of development is scalability. With central code repositories, we can have mirrors and load balanced servers all over the world grab a copy of the latest version every (day/hour/on demand/etc) and be confident that they are identical.

The bottom line is version control is being used by big development teams. If you want to be a professional, you need to be using the tools that professionals use. Your knowledge increases your marketability as a professional developer. Version control is also something that can legitamately help us in our SDLC, and allows teams to simultaniously work on multiple projects securely at radically different points in time, without making any changes to our production servers with great organization and ease.

Stay tuned for day 3.

//Info
Revision Control:http://betterexplained.com/articles/a-visual-guide-to-version-control/

// VCS
Subversion: http://subversion.tigris.org/
Perforce: http://www.perforce.com/perforce/products.html
Git: http://git-scm.com/
CVS: http://www.nongnu.org/cvs/