Last Wednesday I came to work to find my workstation had died overnight — and upon reboot, it failed to mount or fsck the root partition.  Unfortunately it seems my disk has seen enough service and was having failed reads:

Jun 19 08:51:39 atlas kernel: [ 1324.948428] sd 3:0:1:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE,SUGGEST_OK
Jun 19 08:51:39 atlas kernel: [ 1324.948434] sd 3:0:1:0: [sdb] Sense Key : Medium Error [current] [descriptor]
Jun 19 08:51:39 atlas kernel: [ 1324.948442] Descriptor sense data with sense descriptors (in hex):
Jun 19 08:51:39 atlas kernel: [ 1324.948446]         72 03 11 04 00 00 00 0c 00 0a 80 00 00 00 00 00
Jun 19 08:51:39 atlas kernel: [ 1324.948463]         00 03 fa 3f
Jun 19 08:51:39 atlas kernel: [ 1324.948470] sd 3:0:1:0: [sdb] Add. Sense: Unrecovered read error - auto reallocate failed
Jun 19 08:51:39 atlas kernel: [ 1324.948519] ata4: EH complete
Jun 19 08:51:39 atlas kernel: [ 1324.950250] sd 3:0:1:0: [sdb] 156250000 512-byte hardware sectors: (80.0 GB/74.5 GiB)
Jun 19 08:51:39 atlas kernel: [ 1324.950919] sd 3:0:1:0: [sdb] Write Protect is off
Jun 19 08:51:39 atlas kernel: [ 1324.954928] sd 3:0:1:0: [sdb] Write cache: enabled, read cache: enabled, doesn’t support DPO or FUA

After getting a new disk and the machine back operable, I’ve now plugged it in and trying to recover it.  Mounting fails, as it seems there are read errors in the journal itself:

Jun 19 08:35:39 atlas kernel: [  364.686504] EXT3-fs: INFO: recovery required on readonly filesystem.
Jun 19 08:35:39 atlas kernel: [  364.686508] EXT3-fs: write access will be enabled during recovery.
<errors like above>
Jun 19 08:36:08 atlas kernel: [  393.492868] JBD: recovery failed

It seems somewhat logical that it might be common to have physical failures in the area of the disk where the journal lives.  In my case, it seems the unreadable part(s) of the disk are all within the journal.  I ran debugfs on the volume to find that I could read all sorts of things on the disk — so I just needed to tell it to skip reading the journal and mount the disk anyway.

1. Before you remove the journal, you need to remove the needs_recovery flag from the volume.  You’d think this is possible with tune2fs, but it doesn’t seem so.  So you do it with debugfs:
   

   debugfs -w -R “feature ^needs_recovery” /dev/sdb1

2. Then remove the journal, forcibly:
   

   tune2fs -f -O ^has_journal /dev/sdb1

3. Now, go ahead and mount your volume as ext2:
   

   mount -t ext2 -o ro /dev/sdb /mnt/disk

Voila!  You’ve now nuked your journal, and marked your volume ready for mounting.  It’s possible that it has inconsistencies and needs a fsck, but I mounted anyway and was able to recover everything without failure.

I learned this trick from an Ubuntu forums post at http://ubuntuforums.org/archive/index.php/t-953279.html

Wowzers, what a great time! Was intending to write about each day individually, but that’s certainly out of the question now. GUADEC was a great experience and I’m hooked on the community! A big thanks to everyone who helped put GUADEC ‘07 together.

My favorite bits:

• Staying at the hostel. It was a helluva long walk and really sucked on the rainy days, but it was a new experience. Forced random strangers to hang out and I met some cool people. Wish I would’ve gotten the names of the two guys running the hostel; they were very laid back.
• There were some killer keynotes; most importantly:
  • Alex’s presentation of PyroDesktop. I don’t take much stock in the comments of folks who don’t like this technology. It’s something we’ve never seen before, and I think the coolest part is the potential to draw a new crowd of developers to the desktop! Alex, Chris & company: keep up the good work!
  • Havoc and Bryan’s presentation of the Gnome Online Desktop. Super sweet guys. Everyone loves integration. Now, lets see it happen!
  • Doc Searls’ closing keynote. Doc is a funny man. “Now bend over and give me some content!”
• And my favorite sessions:
  • Larry’s presentation of F-Spot and the next wave of development. F-Spot rocks!
  • Telepathy and Tubes. You’ve heard the raves of others; I don’t need to spell it out here.
  • Joe’s presentation on Beagle and metadata joyfulness. Joe is a funny man.
  • Kudos to Jimmy K for holding his own during his talk on the new main menu development.
  • The Geodata standards project — creating a totally free database of mapping information.  Cool!
• The Walkabout.  What a great bar.

Many thanks to all the folks who made going out and drinking absolutely hilarious, including Gabriel, Aaron, Michael, Aiden, Rodney, Toms, any anyone else I’ve missed!

Wowzers, what a day!

• Jono’s talk was excellent.
• Video taped the morning sessions in Recital; the F-Spot presentation was great. Larry is working on some sweet ass stuff. Sync’ing our photos online — integration with web services — I think a common theme is emerging!
• Hacked on libgphoto2-sharp and Banshee. Frustrated like no other. gPhoto doesn’t have any magic support for the lame underlying case-sensitive-insensitive FAT32 filesystem. I guess I kinda figured this would Just Work. I’ve been fighting with this for six months and sure enough, I just need to hack 50 lines of code.
• Telepathy: it’s tubular, dude!
• Too many metas: Joe, thank you for taking care of my metadata. You need to publish what you want from us software hackers so we can help take our desktop to the next generation.
• Havoc’s presentation was great. Full integration of Gnome and the web will make our desktop as useful as possible. Web services are all the rage these days and we’re all dying for that One Way To Link Them All…and I think this is the right approach. Bind our online lives with an online desktop and you’ll get my Mom to switch to Linux.
• To compliment Havoc’s talk (not just because Gabriel is awesome) but Gabriel is right: we need a Free Service License. Havoc mentioned it, and Gabriel has already been thinking of it for a week.
• Been meeting a bunch of people; particularly many of the Novell hackers I’ve heard of and met online several times.
• Went and drank the free beer, and the nice cheap doubles of Southern Comfort…on an empty stomach. Ended up back at the hostel relaxing and eating a really nasty fish ‘n chips.  No, that’s not proper british fish ‘n chips…it was deep fried with french fries.  So greasy it made me nearly sick this morning…

Still grumpy about libgphoto2.  Going to see about fixing it in libgphoto2-sharp, and then push the changes down to libgphoto2 proper.

GUADEC continues!

So I arrived in London around 6 AM; everything went quickly.  Plane ride was fun, albeit quite cozy. Took the paddington express train, which is super slick, jumped on the underground to Euston, and wandered around for a while until my train up to Birmingham. The train ride up was so much faster than I had expected — we don’t have quick, cheap trains like that in the US! By the way — if you want a fun bit of adventure when travelling…don’t bring a map of your destination city with you. Half the fun is wandering around (looking quite touristy with my huge bag) and trying to figure out where the heck I am. Then buy a map :)

I’ve been meeting up with a number of GNOME folks that I’ve known over the last year and a half online, and it’s been a great experience. It’s kinda cool to have a team of Banshee hackers and supporters all roaming around. So many ideas! The hacking itch needs to be scratched this week. Big time.

So rather than turn up and go to sessions, what did I do? I walked across town with a bunch of the guys and watched the big football match. It was a blast! I went thru about 3 sets of batteries taking pictures, which I’ll post soon. Lots of funny action shots of people missing the goal. Hilarity.

Then of course, went out and got some beers and the fixings for some good ‘ol homemade pizza. Brought it back to the hostel where we’re staying at and made some pizza! Thanks to Aiden for helping us out and making it a lot of fun. Combine that with some beers, and then meeting up with a few folks later at a bar, and it was a great evening.  Also managed to give away an Ubuntu live cd to one of the guys working at the hostel — he was quite curious about free software and wanted to give it a try.  We didn’t even have to try to “sell it” — he asked for it!

So, I’ve recently converted my laptop to Gentoo, and before I even had the base system merged in, I thought I should research NetworkManager. I was running Slackware 10.2 on the lappy, and I had never considered NetworkManager due to the fact that I’d probably have to hack at it until it wasn’t Slack anymore. The wonderful Google led me to various blogs of attempts (some working, some failing) and various mailing lists, usually of obscure issues with NetworkManager or dhcdbd not behaving. I also ran across Gentopia, hosted on the Gentoo Experimental project’s site. The Gentopia site itself isn’t anything much to look at…covered in spam and with tons of missing and outdated pages (makes me want to hook up with the developers and help set up some security). Regardless - the Gentopia overlay is there and it fine, so I went with it.

When I finally got everything merged in and logged into my Gnome session, it took me some time to get things really rolling. After poking around, I learned that NetworkManager on Gentoo doesn’t just “take care” of everything and make all the existing network scripts obsolete (like I was thinking it might do) - it actually just detects devices and calls the proper /etc/init.d/net.eth* script to make it go. I didn’t realize this - but once I did, things started to roll!

TIP #1: start your net.eth* scripts _after_ NetworkManager is loaded - this seems to make things work right for me. I fought NM up and down trying to just get it to grab a DHCP IP off my wired network and I couldn’t figure out why it wouldn’t. Turns out that dhcpcd was running for eth0 - and when NM requested an IP over the interface, naturally the interface went to shit and lost it’s IP.

TIP #2: get your interfaces all working properly _without_ NM running. When I tried to get it to work with my wireless card (Cisco Aironet 350), it would recognize that I had a wireless card plugged in but would show no available networks and just scan all day. Stop the NM & dhcdbd services, and tinker with your /etc/conf.d/wireless config until your card is recognized automatically on plugging in and automatically associates correctly and grabs a DHCP IP. If your device isn’t working w/o NM, odds are low that it will work with NM. My biggest conflict is that my card seems to have some memory of an old network (University network) that I used to connect to and defaults to that ESSID. Thus, I have to override it in my /etc/conf.d/wireless.

And some lovely command line output, for proof :)

So…I will probably post more here when I learn more about NM and how it works with Gentoo…but in the mean time, consider trying it out yourself Gentoo’ers!

….in unrelated news, it turns out that you can drag ‘n drop images from wordpress’ inline image uploader, but it’s a really bad idea. Firefox + WP allows it to happen, but the images all come up with broken links as they loose the site’s prefix. Turns out this isn’t a feature of the inline uploader, but just an “unhandled possible feature”. Argh.

Last night the addition of MTP support to Banshee landed in CVS HEAD. Yes, ladies and gentlemen, it’s true! The days of command line interfaces for your snazzy new Creative, iRiver, etc. player are nearly over. All you have to do is TRY IT! Just grab Banshee from CVS and run autogen.sh with the –enable-mtp flag. Oh and yes, be sure you’re running libgphoto2 from SVN trunk with the C# bindings installed. It’s that easy! I’ll be publishing more “how to get started” docs soon.

Yes, your Creative Zen Micro (or other supported MTP player), is ready for use.

Bug reports, feedback, suggestions, hate mail should be either filed on the GNOME Bugzilla under product Banshee, component MTP, or emailed to me.