Note: I’ve written on this topic before, but thought the subject warranted further more detailed discussion and a more comprehensive and up-to-date set of benchmarks. Hence, this post and this presentation. Enjoy.

The SPL, or Standard PHP Library, is an often overlooked extension in the PHP core. It first came on the scene in PHP 5 and a variety of iterators constituted the majority of its initial offerings. Though the iterator offerings were expanded in PHP 5.3, the particularly interesting additions to the SPL were several specialized data structure classes, the foundational concepts for which originate in the field of computer science. In this post, I will provide an overview of these new classes and explain why and when they should be used.

Arrays

While PHP has several data types, the ones that likely see the most frequent and varied use are strings and arrays. They are the proverbial duct tape and WD-40 of PHP, respectively. Like arrays, SPL data structure classes are used to store composite (i.e. non-scalar) data.

Now, that’s not to say that every instance of an array in existing codebases should be replaced with an SPL container object. There are cases where it’s appropriate to use one over the other. Knowing the difference requires an understanding of how arrays work.

Within the C code that makes up the PHP interpreter, arrays are implemented as a data structure called a hash table or hash map. When a value contained within an array is referenced by its index, PHP uses a hashing function to convert that index into a unique hash representing the location of the corresponding value within the array.

This hash map implementation enables arrays to store an arbitrary number of elements and provide access to all of those elements simultaneously using either numeric or string keys. Arrays are extremely fast for the capabilities they provide and are an excellent general purpose data structure.

Fixed Arrays

In contrast to arrays, SplFixedArray functions more like C arrays or Learning the Java Language > Language Basics)" href="http://java.sun.com/docs/books/tutorial/java/nutsandbolts/arrays.html">Java arrays than PHP arrays. The maximum number of elements that it may contain is specified upon instantiation. While it is possible to change it later via the setSize() method, this negates the performance advantages of using it: because its size is fixed, it doesn’t need to use a hashing function to resolve the position of elements within the array. It makes sense to use fixed arrays when the number of elements to be stored is known in advance and the elements only need to be accessed by sequential position.

SplFixedArray implements the Iterator, "/>

Truncated by Planet PHP, read more at the original (another 28336 bytes)

"Countinous Integration is about preventing your developers from burning in Integration Hell" - @s_bergmann

Integration Hell

My code is perfect, yours is pretty good and then Martin...well, he's a fucking idiot. (lost where he was going with this story). D=

Team member should integrate their work frequently. This reduces errors when managing multiple team members. The value (ROI) is in:

• reducing risk
• reduce repetitive processes
• generate deployable software
• enable better project visibility
• establish greater product confidence
• helps with late discovery of defects
  • regression testing
  • test coverage
• prevents low quality software
  • coding standard adherance
• prevents lack of deployable software

This practice focuses on software design that uses unit tests to prevent and detect defects. This all significantly works to provide significantly better quality software.

How do we implement it

We need build automation.

You start by identifying repetitive processes (running tests, analyzing source code, packaging, deployment). You want to make these processes a non-event.

Static code analysis

Lines of code gives a text-based metric for code size.

• Lines of code (loc)
• comment lines of code (cloc)
• non-comment lines of code (ncloc)
• executable lines of code (eloc)

Code duplication. Is it textually identical? token for token identical? functionally identical? Dupicate code contradicts code reuse. Co-Evolution of clones hinders maintenance. There is a tool called PHPCPD (a copy-paste detector). Highest they've seen is 16% duplication in a code base of 5 million lines of code.

code complexity

Cyclomatic complexity counts the number of branching points such as if, for,foreach,while,case,catch,&&,"",ternary operatory. NPath Complexity counts the number of execution paths. Higher complexity leads to more errors and makes testing harder.

You can analyze code with "sniffs".

build automation

• apache ant
• gnu make
• phing
• rake
• shell scripts

Reviewing some of the tools that got covered in another talk I blogged about. If anything new/interesting is added, i'll list it. @s_bergmann recommends Hudson (http://hudson-ci.org). Apparently very easy to use and is used by some big names like Sun.

Related articles by Zemanta

• PHP Code Review (slideshare.net)

<script src="http://static.zemanta.com/readside/loader.js" type="text/javascript">

Image of EliW

@eliw starts off laying down his street cred. He'll be covering how to control the process of version control. As we've been going over the past week, use version control. The talk will be focusing on subversion as the technology but the talk will be dealing with the higher level concepts.

Basic Version Control Terminology

• commit/check-in
• branch
• tag
• trunk
• merge

Subversion thinks in terms of a directory structure. Projects are subdirectories of a repository. The mainline (trunk) is a subdirectory of the project. Branches and tags have parallel directories.

Your release management should come up with general rules that you apply.

• will you allow intermediate (non-working) checkins?
• where should you check code in?
• are there places that as less controlled?
• how does this flow into releases?
• what about tags vs branches vs trunks?

I'm not seeing a reason to allow intermediate (non-working) checkins, that seems like a recipe for disaster.

No matter what your style of management, your trunk should contain the 'core' codebase. Branches are uses to segment off areas of responsibility. Tags should only be used to mark a specific state of code, a release.

Different Branch methodologies (3 main styles)

• stage branches
• feature branches
• release branches

With stage branches, all work is commited against the trunk. When you're ready for a release, you merge into staging then after testing, you merge into production. The catch with this strategy is bringing your changes back to trunk. No parallel work, old patches and room for error with this strategy. There is no way to patch old releases. This setup is about moving forward constantly which works for websites but if you have clients on different versions of an application, things will become a bit hairy. Forgetting to merge things back can cause errors.

With feature branches, all new work you are doing, is done in its own branch. You merge the branch back into trunk after you've tested it. Trunk then is tagged as needed for phases (for testing/QA,Releasing,etc). Parallel work and long scale work become easy. Downside is that you're often creating branches (depends on your VCS), there is a lot of merging, no old patches and fixes are complicated.

So far my current release system seems to be a combination of stage and feature branching. I'll tell you right now that it has been a pain in my ass. Hopefully the next strategy will be my...oh yeah, there is no magic bullet.

With release branches, all new work done on trunk and when ready for realease, creat a versional branch (/branches/v3.0). You can test against the branch and and make bug fixes against the branch. You then tag the branch with a versional tag for release. One of the big pros is that the maintenance work is easy. It is OK with long scale work. There's some parallel work and very little merging. The only time you merge is when you do a bug fix on the branch, it has to be merged back to trunk. Cons include Branch/Tag creation. The biggie is that this assumes a single goal.

Options for pushing code live:

• have a script
• handle multiple machines
• use for all phases staging/testing
• have a rollback procedure
• multiple ways to accomplish
• incorporate everything together
  • Services, DB, PHP, etc

This is a great talk so far, I'm having a lot of great ideas for my release structure but something tells me that where I work now won't let me put this in place. D=

Live check out of the server is very simple. Big drawback is conflicts, hard to automate & rollback. As of a year ago,

Truncated by Planet PHP, read more at the original (another 1697 bytes)

Image via Wikipedia

We're starting with a map of the world showing the 24 major timezones. Timezone changes (daylight savings times) make things tricky. The abbreviations for timezones aren't enough to determine the user's timezone. EST can mean different things.

The 64-bit signed integer used internally provides more than enough time for us to use (+/- 90 billion years if I heard correctly). strtotime() and functions with timestamps have been replaced with classes (such as new DateTime()).

The bundled timezone database has 564 zones so far that isn't dependent on timezone abbreviations. They have the format Continent/Location or Continent/Location/Sublocation like Europe/Amsterdam. Updated database is released 20 times a year. Some changes are very sudden. Basically, you're timezone database is probably outdated.

You can set the default timezone with a function or the DateTime object. PHP guesses it in the following order:

• date_default_timezone_set()
• TZ environment variable
• php.ini's date.timezone setting
• system's rendering of timezone abbreviation.

PHP 5.3 will not guess and will complain.

You can parse strings with the date_parse() function.

The DateTime object seems very powerful. It allows you to modify times and will account for time zone changes for some of its other uses.

I'm thinking I'll have to wait for slides, lots of code examples that I'm having a hard time figuring out how to explain in a post. In regards to relative time, remember that time always moves forward. If the date is Wednesday, asking for Tuesday will get the next Tuesday. Yeah, the code examples and output are probably better but will have to wait until I can link to them.

<script src="http://static.zemanta.com/readside/loader.js" type="text/javascript">

Image of Rob Allen

Trying to record @akrabat's talk about zend_form but didn't get good camera placement. Won't be able to put up the video I do have until after Amsterdam. Will need to get a voice recorder instead.

Filters are destructive. This is important to know that. Validators only reject data. Decorators are used for form rendering. Apparently I'm not the only one confused by it because Rob Allen said we'll go over that extensively. Zend_Form decorators are a combination of Decorator and Strategy Pattern.

Creating form is as easy as extending the form and sticking things in init(). Form elements implements the fluid interfaces so you can chain methods. When creating an enlement, the name you pass into the constructor will become the name of the element. Validators can be added with the string name or the class itself. After making the element, don't forget to add it to the form. Caveat on Submit, setLabel() will change the value. Be sure to setIgnore() to true because you won't care about the value of the button. This is a new hint/trick that I had missed in the past. This is helpful.

Now we're going over an action code example. Basic stuff you can find from the manual on the Zend Framework site. View examples from the site as well. You can echo out the form only or by element in your view. "Under no circumstances display the email address validator to your customer." How do you fix it? "Translate" the errors. Create a translate object. Set the translator to the form via the form's setTranslator() method. If you have a lot of forms, you can use the static setDefaultTranslator() method.

On to the meat of the talk (in my opinion), Decoration. =D

Default element decorators

1. ViewHelper
2. Errors
3. Description
4. HtmlTag
5. Label

Order is very important. They render inside out. Even though the label is last, it prepends itself before the element. When things don't go correctly, it's ALWAYS because you fucked up the order. Adding a required asterick can be set by retrieving the decorator via getDecorator() and set the option requiredSuffix.

Using LI Instead is clearing the decorators then adding the new ones. You render the  label then the html tag decorators. That's the difference. is the last 2 in the above list is flipped. (I'll try to add code examples later or a link to the slides if they're available.)

On to form decorators.

Default Form Decorators

1. FormElements
2. HtmlTags
3. Form

Once you understand how things work, they're relatively consistent. Again, for lists, clear the decorators in the init and add them with your tag using ul.

Going over custom elements. Demo is a YesNo element to set radio multiOptions. Setting the separator on this element with a space will get rid of the <br/> tag that is rendered by default. A ThumbsUp decorator is being used as an example to overriding the render. One interesting I just learned is if there are messages present, that is because a validation failed. That makes sense and can be used for finding out if there are errors.

While it seems like it is complex to do via the object when you could do html, the validators and filtering are really great for getting good data. This is what makes most of the 'work' with Zend_Form worthwhile. Really great talk, being here for this talk justifies my trip out here. I'm hoping all the

Truncated by Planet PHP, read more at the original (another 1690 bytes)

Josh Holmes is giving the keynote and is very insistent on enthusiasm. Talking about simple projects and used twitter as an example. Twitter may have been been written in a weekend by most of us, it is successful because it filled a niche. Seems like part of the talk will be analyzing why we say "I could of wrote that in a weekend" with an air of disdain.

Looking at the definition of simplicity. First few definitions talk about foolishness and naiveté but the last definition is where we focus on "clarity of expression" and "Austerity in embellishment."

Analyzing the "proper way to do things" as a method of keeping control. Average project doesn't finish. Lasts a year, 18 month, "at least 2 months after deadline". "We end up saving our problem like Wile E. Coyote."

Just because you throw more people at a problem, doesn't make it go quicker. "Nine women can't birth a baby in a month." Referencing concepts from  The Mythical Man-Month and No Silver Bullet. PHP 5.3 isn't a magic bullet that will save the world.

Analyze if that new shiny will really make a difference in the project before chasing it. Either way, we should try to get back to basics. A programmer doesn't just use a set of tools but understands what is going on underneath the hood. Are you a programmer or script kiddie? We need to encourage the script kiddies to get back to fundamentals.

If an idea can not be expressed simply, then you're doing it wrong. How do you explain what you do to the average person?

Do you understand your craft enough to explain it simply and do it simply. This leads to looking at your toolset. Do you use your framework as a hammer without considering other frameworks?

The trick is to solve today's problem while preparing for the future. The death of a startup is tends to happen because they were not prepared for when their project takes off.

Basically, try not to over-engineer, prioritize the features by ROI, look at the usability and most importantly, have tests. Josh's call to to action is to go to war against complexity. Look at your engineering practices, do you understand your processes?

Edit: this was published late since I forgot to hit publish before moving to another session, I'm correcting the timestamp to reflect when it should've been published.

<script src="http://static.zemanta.com/readside/loader.js" type="text/javascript">

When I was growing up in India, we had to take “computer programming” courses in 5th grade.  At that time they taught us a language called Logo . Making a turtle draw concentric circles was easy, so my friends and I figured out how to get out of what was then an IDE, into qbasic so we could play Gorilla.

Once we figured out how to get out of Logo, word grew throughout the school that some of us had figured out how to do it, other kids asked us how to do it.  Then, one of us had the idea of making the computer beep every time the teacher said “sheet”.. (you have to be North Indian, be able to identify a south indian accent, and be a 5th grader to get the joke.. trust me).

Once the beeps got old and annoying, a girl asked if it was possible to make the computer “sing”. long story short.. we figured that part out just in time for christmas, and most of the computers were singing Jingle Bells.

So in summary, we had 5 steps.

1. Phase 1 – Get out of Logo (notice that you have an itch)
2. Phase 2 – Get out of Logo (itch scratched, tell people about it)
3. Phase 3 – People ask you to help them get out of Logo (lunch money)
4. Phase 4 – Censor the teacher (show students what else can be done)
5. Phase 5 – Girl (High value customer) asks for a feature (get the girl) (to my wife, I don’t even remember her name.. seriously)..

Now, I’m 31, and there’s a new girl I’m trying to get. I need to do what I did in 5th grade, to get this girl as well, I don’t know why I’ve been doing everything but that.  I’d be willing to bet, I can do it in 4 steps now.

Two weeks after the Month of PHP Security closes Stefan Esser will teach an advanced PHP security course at the SyScan Singapore security conference. The course will cover advanced techniques to audit PHP applications for security problems at source code and bytecode level. Don’t miss your chance to learn howto find PHP application security vulnerabilities from our PHP security expert himself.

Advanced PHP Auditing at Source and Bytecode level

This course will teach students advanced methods and techniques for PHP applications audits at source code and at bytecode level. The students will get to know the most common PHP security problems and how to find them at source code and bytecode level. Throughout the course several free and open source software tools will be introduced and used in order to visualize application structure, find security problems with static and dynamic analysis on source code and bytecode level and also to break PHP bytecode encryption.

Student Pre-requisite:

Ability to read, understand and develop PHP code.

Software Requirement:

Required software will be delivered in form of a VMWARE Ubuntu Linux installation.

Hardware Requirement:

Laptop Computer

Course Outline:

Source Code Auditing
——————–
Introduction to PHP Source Code Audits

• What to look for
• How to look for it

Common and lesser known Vulnerabilities

• How they look like
• How to find them

Visualization Techniques

• Code Coverage
• Callgraphs
• Classgraphs
• Function Traces

Static vs. Dynamic Analysis

Tools

• Grep + regular expressions
• Xdebug
• Bytesuite
• Dot / yEd

Bytecode Level Auditing
———————–
Introduction to the Zend Engine

Instruction Set of the Zend Engine/PHP Bytecode

• Important PHP Bytecode instructions
• How PHP Vulnerabilities look at Bytecode Level

PHP Bytecode Visualization

• Code Coverage at Bytecode level
• Callgraphs
• Code Flow Graphs
• Classgraphs

PHP Bytecode Encryptors

• How they work
• Weaknesses
• Decryption

PHP Bytecode Decompilation

Static and Dynamic Analysis

• Collecting variable types
• PHP Tainted Mode
• Data flow analysis

Tools

• Dot / yEd
• Xdebug
• Vld
• Bytekit
• Bytesuite
• PHPDecompiler

It was discovered that access control to the [php] bbcode which allows executing PHP code is wrongly implemented in e107. This allows unauthenticated users to execute arbitrary PHP code easily.

Affected versions

Affected is e107 <= 0.7.20
MOPS-2010-111
MOPS-2010-112

Risk

Highly Critical.

Credits

The vulnerability was discovered by Stefan Esser.

About e107

e107 is a content management system written in PHP and using the popular open source MySQL database system for content storage. It’s completely free, totally customisable and in constant development.

Detailed information

Within e107 there is a special bbcode [php] that allows executing arbitrary PHP code. Because it is rather dangerous the configuration of e107 disables access to this bbcode for all users normally. The admin of a e107 site can activate it on demand for certain user groups.

While auditing e107 it was discovered that the access control checks are not within the core of the bbcode parser but in some outer functions that call the bbcode parser. An example for such a check is seen below.

    function post_tohtml($text, $modifier = true, $extra = '') {
        ...

        //If user is not allowed to use [php] change to entities
        if(!check_class($pref['php_bbcode']))
        {
            $text = preg_replace("#\[(php)#i", "[\\1", $text);
        }

        return ($modifier ? $this->tohtml($text, true, $extra) : $text);
    }

This code shows that there is most likely no access check to [php] in the tohtml() method (and indeed there is not), because it is checked outside of it. This means user input should never be allowed to reach the tohtml() method directly because otherwise it will result in a remote PHP code execution vulnerability.

However when looking at the code it is possible in several different places for user input to reach tohtml() directly. One example is within the toEmail() method. (NOTE: this is only ONE example)

"/>

Truncated by Planet PHP, read more at the original (another 8685 bytes)

PHP’s iconv_mime_encode() function can be abused for information leak attacks, because of the call time pass by reference feature. This vulnerability also demonstrates that fixing zend_parse_parameters() is not enough to kill some of these vulnerabilities.

Affected versions

Affected is PHP 5.2 Affected is PHP 5.3

Credits

The vulnerability was discovered by Stefan Esser during a search for interruption vulnerability examples.

Detailed information

This vulnerability is one of the interruption vulnerabilities discussed in Stefan Esser’s talk about interruption vulnerabilities at BlackHat USA 2009 (SLIDES,PAPER). The basic ideas of these exploits is to use a user space interruption of an internal function to destroy the arguments used by the internal function in order to cause information leaks or memory corruptions. Some of these vulnerabilties are only exploitable because of the call time pass by reference feature in PHP.

After the talk the PHP developers tried to remove the offending call time pass by reference feature but failed. The feature was only partially removed which means several exploits developed last year still worked the same after the fixes or just had to be slightly rewritten. One of these exploits exploits the iconv_mime_encode() function.

PHP_FUNCTION(iconv_mime_encode)
{
    ...

    if (zend_parse_parameters(ZEND_NUM_ARGS() TSRMLS_CC, "ss|a",
        &field_name, &field_name_len, &field_value, &field_value_len,
        &pref) == FAILURE) {

        RETURN_FALSE;
    }

    ...

    err = _php_iconv_mime_encode(&retval, field_name, field_name_len,
        field_value, field_value_len, line_len, lfchars, scheme_id,
        out_charset, in_charset);

Similar to the other interruption information leak vulnerabilities zend_parse_parameters() is used to retrieve up to three arguments into local variables. The only difference here is that the last parameter is an array unlike in all the other examples. For the two string parameters the same rule applies: copying the string pointers into local variables makes them vulnerable to any modification to the string ZVALs by an interruption attack. And again a __toString() method could be used to achieve that within zend_parse_parameters(). This is a repeating pattern that was also recognized by the PHP developers. Therefore their idea to fix all those information leak interruption vulnerabilities was to make the __toString() attack impossible. However iconv_mime_encode() is an example that shows why this is not sufficient. In order to re

Truncated by Planet PHP, read more at the original (another 22382 bytes)