Character Encoding at The Conversations Network

Here’s a post that will be of interest only to that small percentage of Blogarithms readers who are struggling with character-encoding issues in the world of PHP/MySQL. But to those of you who fall into that category, this may be helpful. In the process of preparing for the launch of our first series in French, I’ve been working my way through this issue. Although we haven’t tested everything and I expect there will be bugs, here’s what I’ve learned/decided so far.

  • We store everything in the CMS database as utf-8.
  • Nearly all CMS strings are stored in the database as simple utf-8 without HTML entity encoding. HTML is not allowed in most CMS fields.
  • One can check MySQL’s charset at its phpMyAdmin main page: /main.php
  • Our Long Description field (shows.description) is the one CMS field that has special handling:
    • Many HTML elements are allowed in this field.
    • The TinyMCE editor enforces the HTML elements rules, eliminating those that are not allowed.
    • The TinyMCE editor is configured to HTML encode <,> and & on output. The encoding is invisible unless the user activates HTML mode.
    • All characters are still utf-8 encoded as elsewhere.
  • All HTTP responses include a Content-Type header specifying utf-8 character encoding. (“AddDefaultCharset utf-8” in http.conf.)
       Content-Type text/html; charset=utf-8
  • In generated RSS feeds all strings are HTML entity encoded (<>& only) during feed generation.
  • We also convert various strange characters such as those that may have been copied/pasted from a Microsoft Word document. Current such transformations include
    • slanted single and double quotes
    • various em and long dashes
  • All generated RSS feeds use utf-8 character encoding:
       <?xml version='1.0' encoding='utf-8' ?>
  • All HTML pages use utf-8 character encoding:
          <meta http-equiv="Content-Type"
                charset=utf-8" />
  • Immediately after opening a database connection and before any other query is performed, the following query is run to ensure that MySQL performs no character-set transformations:
       SET character_set_results = 'utf8',
           character_set_client = 'utf8',
           character_set_connection = 'utf8',
           character_set_database = 'utf8',
           character_set_server = 'utf8';
  • All form elements are (or at least should be) written as follows. It’s not clear what a browser will do with this, for example if the user pastes into an <input> field text that has been copied from a Microsoft Word document with non-utf-8 encodings.
       <form accept-charset='utf-8'...

The Conversations Network Survey — Features

More our 2010 survey of registered members of The Conversations Network:

Here’s how many said they found features “somewhat” or “very” valuable:

  • Our 5-star ratings system: 67% (65% last year)
  • Personal recommendations: 58% (60%)
  • The Conversations Network Newsletter: 55% (615)
  • Personal Playlists: 42% (49%)

When we asked what features were missing from the websites, the most-common answers were suggestions to improve the usability and navigation.

When asked about how we currently announce new shows on Twitter, 36% said “Don’t change anything” while 60% said “I really don’t care!”

CHI Conversations — Annual Survey

Continuing the results of The Conversations Network’s 2010 annual survey…

CHI Conversations (our newest channel):

  • Didn’t know about it: 58%
  • Never listen: 27%
  • Listen occasionally, often or regularly: 15%

How did you discover CHI Conversations?

  • Another podcast: 29%
  • Search engine: 16%
  • Link from another web site: 13%

Satisfaction was consistently “good” (as opposed to “poor” or “excellent”) for  content quality, audio quality, sophistication, diversity and overall.

26% visit the CHI Conversations website at least monthly.

18% subscribe to the channel’s RSS feed.

IT Conversations — Annual Survey

Continuing the results of The Conversations Network’s 2010 annual survey…

IT Conversations

  • Didn’t know about it: 13% (5% last year)
  • Never listen: 8% (5%)
  • Listen occasionally, often or regularly: 79% (90%)

How did you discover Social Innovation Conversations?

  • Link from another web site: 29% (33% last year)
  • Another podcast: 18% (wasn’t a choice last year)
  • Search engine: 23% (21%)

Satisfaction was consistently “excellent” (as opposed to “poor” or “good”) for  content quality, audio quality, sophistication and overall. “Diversity” was ranked as only “good”. Last year IT Conversations was rated “excellent” by all criteria.

We used to publish a weekly list of recently published programs.

  • 61% say they would subscribe
  • 39% say they would not

Asked to rate our current series in order of “interest” out of 19 active series:

  1. Interviews with Innovators (Jon Udell) — most interesting
  2. Tech Nation (Moira Gunn)
  3. Emerging Technology (ETech)
  4. Technometria (Phil Windley)
  5. Open Source Conference (OSCON)
  6. StackOverflow
  7. Web 2.0
  8. IEEE Radio Spectrum

[Note: StackOverflow, recently suspended by hosts Jeff and Joel, has been the leader in terms of listenership.]

We asked listeners to tell us what other events we should include. The 66 responses included the following trends:

  1. TED, PopTech! and SuperNova (we can’t get the rights)
  2. vendor-specific events (generally too commercial for ITC)
  3. open-source events such as Apachecon

Social Innovation Conversations — Annual Survey

Continuing the results of The Conversations Network’s 2010 annual survey…

Social Innovation Conversations

  • Didn’t know about it: 30% (16% last year)
  • Never listen: 18% (17%)
  • Listen occasionally, often or regularly: 52% (67%)

How did you discover Social Innovation Conversations?

  • Link from another web site: 31% (same as last year)
  • Another podcast: 18% (wasn’t a choice last year)
  • Search engine: 16% (13%)

Satisfaction was consistently “good” (as opposed to “poor” or “excellent”) for all criteria: content quality, audio quality, diversity, sophistication and overall. (Same as last year.)

“Eric and Alana’s Conversations Update” (our newsletter):

  • 71% don’t receive it (65% last year)
  • 11% receive it but don’t read it (10%)

Of those who do read it (18% this year, 24% last year):

  • 16% say it’s very valuable (19%)
  • 56% say it’s of moderate value (56%)
  • 28% say it’s of little value (25%)

Respondents gave the highest priority to the following topics (in order):

  • Education
  • Environment and energy
  • Social entrepreneurship

The topics given the lowest priority were animal welfare and religion. Common themes from the free-form answers included (in order of frequency):

  • Technology in social innovation
  • Human rights
  • Sustainability
  • Non-profit government and management

We asked, “What other [resources] would you like SIC to bring to your attention?”

  • Blogs, web sites, podcasts: 73% (83% last year)
  • Lectures: 60%
  • Conferences: 51%
  • Magazines: 32%
  • and to a lesser extent: newspapers, workshops and week-long programs.

Regarding our SIC Facebook page:

  • 43% don’t use Facebook
  • 49% said “Don’t change anything.”

56% (54% last year) were aware that our SIC channel is produced in partnership with the Stanford Center for Social Innovation.

84% told us that the content of SIC was relevant to their country. 58% said they were in the U.S.

The Conversations Network — Annual Survey Results

A few weeks ago we published the results of the annual survey of members. Beginning with this post, I’ll be blogging the results of the larger survey of all registered members of The Conversations Network. While is our 14-month old site for finding and sharing all spoken-word audio and video, The Conversations Network survey covers our proprietary channels: IT Conversations, Social Innovation Conversations and CHI Conversations. As of this date, the new survey has been completed by 302 members versus 461 last year. (The new survey is still open.) Note that last year’s data are shown in [square brackets].

How do you listen to The Conversations Network?

How do you listen to podcasts?

  • Computer: 56% [57%]
  • Portable device: 76% [75%]
  • Burn to CD/DVD: 7% [6%]

Do you subscribe to one or more of our RSS feeds?

  • Yes: 60% [83%] This question was asked in a somewhat different manner last year.

For those who subscribe to RSS feeds, what software or service do you use?

  • Google Reader/FeedFetcher: 25% [25%]
  • iTunes/Mac: 24% [26%]
  • iTunes/Windows: 23% [22%]
  • other: 53% [52%]

Are you an customer?

  • Current customer: 15% [14%]
  • Never been a customer: 66% [72%]
  • Used to be a customer: 19% [14%]

How many programs on The Conversations Network have you heard in the past month?

  • None: 17% [11%]
  • 1: 15% [7%]
  • 2-5: 40% [37%]
  • 6-25: 25% [37%]
  • 26 or more: 2% [8%]

Tomorrow: Social Innovation Conversations

Avoid QuickBooks and Intuit’s Extortion

Here at the non-profit Conversations Network we’ve been using QuickBooks for Mac 2007 since…2007. It’s not a great program but it works. A few weeks ago I  sent my Mac out for repairs and they had to replace the main logic board. When it came back, QuickBooks decided I was using an unregistered copy of the software and asked me to register. Only problem: Their registration program won’t run under SnowLeopard (OS X 10.6). The only choice: Upgrade to QuickBooks 2010. No other options.

I can count on a few fingers the software that forces me to buy a new version when upgrading the OS. I’m still running an antique version of Microsoft Office. Heck, even that stuff from sleazy Adobe still works. And none of these programs died just because I replaced the logic board. Let’s be honest here: If Intuit wanted to support its customers, they’d send out a fix to their registration program so that people who needed to re-install under an upgraded OS could so so. Again: the app runs fine; it just thinks it’s unregistered and therefore stops after 15 executions.

There are a lot of apps and vendors whose software I find annoying (eg, Adobe), but none are as sleazy as Intuit. I sure wish I knew of a viable alternative to QuickBooks for Mac that was suitable for a small non-profit.

p.s.: Officially, there’s no “upgrade” price for QuickBooks for Mac 2010. But the support people quoted me $159 so when I called Sales I held them to that price and they approved it.

Update: Sometimes I forget who reads this blog and my Twitter posts! A few hours after posting the above I got a call from an apologetic product manager at Intuit. Apparently there’s quite a bit of misinformation within his sales and support teams. Here’s what I learned:

  • There’s no reason (other than that misinformation) that the QuickBooks support folks shouldn’t have been able to help me re-register my 2007 version for Snow Leopard.
  • Although some features may not work right under Snow Leopard — I haven’t encountered any of those features — there’s no other reason I shouldn’t be able to continue to use my 2007 version on that version of the OS.

So if your old version of QuickBooks wants to be re-registered because you change your motherboard — I assume they’re locked into the LAN MAC address — don’t let anyone tell you the only choice is to upgrade.

BTW, I haven’t been keeping track, but I *think* the only other app that had to be re-registered after the Mac Pro came back from the shop was the Apple stuff: iTunes, Final Cut Pro, etc. I don’t think Adobe’s Lightroom, Photoshop, etc., required this, but I was so busy getting a week’s worth of stuff transferred from my laptop to the Mac Pro after the repairs, I wasn’t really keeping track.