Seth's Blog

In March I discovered that looking at a graph of my productivity (for the current day, with a percentile attached) was a big help. My “efficiency” — the time spent working that day divided by the time available to work — jumped as soon as the new feedback started (as this graph shows). The percentile score, which I can get at any moment during the day, indicates how my current efficiency score ranks according to scores from previous days within one hour of the same time. For example, a score of 50 at 1 p.m. means that half of the previous days’ scores from noon to 2 p.m. were better, half worse. The time available to work starts when I get up. For example, if I got up at 4 a.m., at 6 a.m. there were 2 hours available to work. The measurement period usually stops at dinner time or in the early evening.

This graph shows the results so far. It shows efficiency scores at the end of each day. (Now and then I take a day off.) One interesting fact is I’ve kept doing it. The data collection isn’t automated; I shift to R to collect it, typing “work.start” or “work.stop” or “work.switch” when I start, stop, or switch tasks. This is the third or fourth time I’ve tried some sort of work tracking system and the first time I have persisted this long. Another interesting fact is the slow improvement, shown by the positive slopes of the fitted lines. Apparently I am slowly developing better work habits.

The behavioral engineering is more complicated than you might think. My daily activities naturally divide into three categories: 1. things I want to do but have to push myself to do. This helps with that, obviously. 2. things I don’t want to do a lot of but have to push myself away from (e.g., web surfing). 3. things I want to do and have no trouble doing. But the recording system is binary. What do I do with activities in the third category? Eventually I decided to put the short-duration examples (e.g., standing on one foot, lasts 10 minutes) in the first category (counts as work), keeping the long-duration examples (e.g., walking, might last one hour) in the second category (doesn’t count as work).

Before I started this I thought of a dozen reasons why it wouldn’t work, but it has. In line with my belief that it is better to do than to think.

I fixed a few problems and eliminated the one Windows-specific function so it can be used with Macs.

The new version is here.

I have put a requested R workspace on my website so that you can download it. The percentile feedback workspace compares your productivity (time spent working/time available to work) today to previous days. When I started using it, I became more productive. Here is an introduction. Here are all posts about it.

This is not for everyone. You need R installed to use it (of course) and you’ll need to know at least a little R. You must edit a function called save.ws so that the workspace is saved in the right place. I have used it under Windows XP.

Warning: This post, written for the Quantified Self blog, has more repetition than usual of material in earlier posts.

In January, after talking with Matthew Cornell, I decided to measure my work habits. I typically work for a while (10-100 minutes), take a break (10-100 minutes), resume work, take another break, and so on. The breaks had many functions: lunch, dinner, walk, exercise, nap. I wanted to do experiments related to quasi-reinforcement.

I wrote R programs to record when I worked.  They provided simple feedback, including how much I had worked that day (e.g., “121 minutes worked so far”) and how long the current bout of work had lasted (e.g., “20 minutes of email” — meaning the current bout of work, which was answering email , had so far lasted 20 minutes).

I collected data for two months before I wrote programs to graph the data. The first display I made (example above) showed efficiency (time spent working/time available to work) as a function of time of day. Available time started when I woke up. If I woke up at 5 am, and by 10 am had worked 3 hours, the efficiency at 10 am would be 60%. The display showed the current day as a line and previous days as points. During the day the line got longer and longer.

The blue and red points are from before the display started; the green and black points are from after the display started. The red and black points are the final points of their days — they sum up the days. A week or so after I made the display I added the big number in the upper-right corner (in the example, 65). It gives the percentile of the current efficiency compared to all the efficiency measurements within one hour of the time of day (e.g., if it is 2 p.m., the current efficiency is compared to efficiency measurements between 1 p.m. and 3 p.m. on previous days).

I started looking at the progress display often. To my great surprise, it helped a lot. It made me more efficient. You can see this in the example above because most of the green points (after the display started) are above most of the blue points (before the display). You can also see the improvement in the graph below, which shows the final efficiency of each day.

My efficiency jumped up when the display started.

Why did the display help? I call it percentile feedback because that name sums up a big reason I think it helped. The number in the corner makes the percentile explicit but simply seeing where the end of the line falls relative to the points gives an indication of the percentile. I think the graphical display helped for four reasons:

1. All improvement rewarded, no matter how small or from what level. Whenever I worked, the line went up and the percentile score improved. Many feedback schemes reward only a small range of changes of behavior. For example, suppose the feedback scheme is A+, A, A-, etc. If you go from low B- to high B-, your grade won’t change. A score of 100 was nearly impossible, so there was almost always room for improvement.

2. Overall performance judged. I could compare my percentile score to my score earlier in the day (e.g., 1 pm versus 10 am) but the score itself was a comparison to all previous days, in the sense that a score above 50 meant I was doing better than average. Thus there were two sources of reward: (a) doing better than a few hours ago and (b) doing better than previous days.

3. Attractive. I liked looking at the graphs, partly due to graphic design.

4.  Likeable. You pay more attention to someone you like than someone you don’t like. The displays were curiously likable. They usually praised me, in the sense that the percentile score was usually well above 50. Except early in morning, they were calm, in the sense that they did not change quickly. If the score was 80 and I took a 2-hour break, the score might go down to 70 — still good. And, as I said earlier, every improvement was noticed and rewarded — and every non-improvement was also gently noted. It was as if the display cared.

Now that I’ve seen how helpful and pleasant feedback can be, I miss similar feedback in other areas of life. When I’m walking/running on my treadmill, I want percentile feedback comparing this workout to previous ones. When I’m studying Chinese, I want some sort of gentle comparison to the past.

Here is another example of the efficiency graphs I’ve blogged about (here, here and here). The line is the current day; it shows how well I’m doing compared to previous days. It goes up when I work, down during breaks. The number in the right corner (“77″) is the percentile of my current efficiency (at the time the graph is made) compared to measurements within one hour (e.g., a measurement at 2 pm is compared to previous measurements between 1 pm and 3 pm).

The blue points come from before I started the feedback; the green points, afterwards. The red and black points are the final points of a day (that is, at quitting time). That the green points are above the blue points suggests that the graphical feedback helped. Here is a better way of seeing the effect of the feedback.

I didn’t expect this, as I’ve said. It is not “the effect of feedback”; before the graphical feedback, I’d gotten non-graphical feedback. It is a comparison of two kinds of feedback.

Why was the new feedback better? Here’s my best guess. It helped a little that it was pretty (compared to text). It helped a lot that it was in percentile form (today’s score compared to previous scores). This meant the score was almost never bad (from the beginning the percentile was was usually more than 50) and yet could always be detectably improved (e.g., from 68 to 70) with a little effort. I wish I could get such continuous percentile feedback in other areas of life – e.g., wwhile treadmill running. I think feedback works poorly when it is discouraging or unpleasant and when it is too hard to improve. When I taught a freshman seminar at Berkeley, I got feedback (designed by a psychology professor) that was so unpleasant I stopped teaching freshman seminars. Because it came only at the end of the term, it was hard to improve — you’d have to teach the class again to get a better score. Moreover, it compared your score to everyone else’s.  I think I was in the lower 50%, which I found really unpleasant. There was no easy way to give feedback about the feedback; maybe it is still in use.

In contrast, I love the feedback shown in the upper graph. Not only does it really help, as the lower graph shows, it leaves me at the end of the day with a feeling of accomplishment.