• Calculating the Differences Between Consecutive Rows with SQL

    Just a quickie. I’m sure this is obvious to the seasoned SQL freaks out there, but it was a new solution for me.

    We were recently trying to figure out a simple query to generate a set of differences in values between sets of consecutive rows in a database table. Basically, the table is full of timestamp snapshots with associated values, and we need to calculate the differences between the values in each snapshot for a time period.

    The simple solution I came up with is to normalize the dates, and then use them as a join criteria with a virtual table and some simple date math. You treat the original unmodified table as one side of the join, and a virtual select of the same table with some date math applied to the normalized dates as the other side of the join. So, for tables with timestamps five seconds apart, you simply pull the date back by five seconds on the virtual table and join the dates together. You end up with a single row joining each row with its previous peer, and you can then do direct math within the row to calculate the differences in values.

    Here’s a simple MySQL script that demonstrates the technique, which will probably make more sense than my explanation.

    CREATE TABLE consecutiveDates (
      valueDate TIMESTAMP NOT NULL,
      value NUMERIC(10) NOT NULL,
      PRIMARY KEY (id)
    INSERT INTO consecutiveDates VALUES (NULL, TIMESTAMP('2009-08-01 10:00:00'), 10);
    INSERT INTO consecutiveDates VALUES (NULL, TIMESTAMP('2009-08-01 10:00:05'), 25);
    INSERT INTO consecutiveDates VALUES (NULL, TIMESTAMP('2009-08-01 10:00:10'), 50);
    SELECT    actual.*,
              IFNULL((virtual.virtualValue - actual.value), 0) AS 'difference'
    FROM      consecutiveDates actual
              LEFT OUTER JOIN (
                 SELECT   id AS 'virtualId',
                          valueDate AS 'actualVirtualDate',
                          value AS 'virtualValue',
                          TIMESTAMPADD(SECOND, -5, valueDate) AS 'adjustedVirtualDate'
                 FROM     consecutiveDates
              ) virtual ON virtual.adjustedVirtualDate = actual.valueDate

    If you have dates that don’t line up perfectly like my example, you can use basic date math/conversion to normalize the dates in each row in to a rounded representation (i.e. nearest five seconds, nearest minute, etc.). With basic SQL grouping, you can then easily smash together rows that are close enough to each other to be combined in to a single row and achieve the same effect as shown in the script.

    This technique is obviously not limited to dates; you can use a similar approach with virtual row IDs in MySQL. By creating an integer session variable representing a row ID, and then selecting/incrementing it with each row in the query, you can apply basic math to the virtual table’s select statement to offset the session variable by -1 and use that as the join criteria.

    Category: Uncategorized | Tags: ,