Yesterday was spent cleaning up, filing expense reports, and doing little random things. FWIW, I had 258 emails in my inbox, although I was checking my email some during that first week; a first pass got it down to 101, and I ended the day at about 10, including what I’d received over the course of the day.

Today…well, there’s some more of all that, but I did print up my notes from the Google Analytics training. I’m hoping for a chance to read through and condense it into a report for my boss and a to-do list. I took all my notes here and have them saved as draft posts; if my notes turn into something shareable (ie, not too much internal info) then I’ll post it here.

Tomorrow I’m planning on doing the same with my Drupal Summit notes, which are already available here in live-blogging form.

All of that, hopefully, should provide some redirection on my projects for the rest of the year and for 2011. During the whole week, I kept realizing all the back-to-basics things that I want to be doing. I’ve spent most of the last five years constantly surging forward into new areas; now it’s time to review, to assess, to clean up and polish. I’m looking forward to it.

(Plus I read Rocket Surgery Made Easy while I was sick, and I’m inspired to do more usability testing!)

not the first experience you want people to have: “Bet you its a duplicate” (bug submission)

value of project is social capital: number of people contributing – harkens back to yesterday’s keynote, and that organizations evaluate OS projects based on number of developers.

community management = negotiation!

everybody’s pointing at webchick & jhodgson

hiring on technical skills, promoting on soft skills. and nobody says that. the big lie around the meritocracy.

the tools for treating this as a science already exist

lower transaction costs for engaging in the community: skills, design, data

a tweet: “the part of open source I dislike the most is [dammit, didn't get to translate, but amounts to I hate it when everyone is wrong and just doesn't use my idea]

from a negotiation perspective this is the worst option

at the core of every conversation: interests (as separate from positions) – why do you want this? underlying motivations. (it’s like doing usability testing!)

the smart-alec comment: I just want you to tell me that I’m smart.

use them to generate options, then what external standards can you use to make decisions. (legitimacy)

often breaks down because relationship and/or community are lacking

in open source communities, relationships are often MUCH weaker. communication also often weak, missing 70% because of lack of visual/audio cues.

“crazy distorted version of this framework”

build this stuff into the tools we use – nudge people to better communication

Inquire – Paraphrase – Acknowledge – Advocate

example of bug report (?) that is ALL advocating

modeling behavior and setting norms

“what should the software have done?” vs “What were you trying to do?/Why did you want to do that?”

and yet there’s all these tools out there

fork as a big threat – but github as a structure allows people to run off and innovate, at an infinite scale

attracting/empowering the lone individual

cooperation vs collaboration challenge

collaboration is very high-touch; open source restrctured process to enable cooperation, going off on your own and then bringing it back into the whole.

example of Firefox add-ons – which sounds a LOT like Drupal module situation

mapping consumption of memory by add-ons: a real issue of cooperation – then providing more data can empower the user to make better choices, which provides useful feedback to addon developers.

discussion of bug vs support

developers are incented to be mean, because they don’t want to waste their time. so talking about changing the culture maybe not the best place to start. look at the incentive structure: what’s driving the behavior?

open source projects don’t have open data portals.

if you want to advocate for drupal, you have to recognize that there are things that drupal is not good at. (in context of bug/issue tracking)

how do we create dashboard systems? shows example of seeclickfix

the hardest people to get data about are the people who leave and never come back.

need to bring real accountability to project/module owners – track wait times, checkins, etc. setting expectations. (would be interesting to combine that with time-limiting responsibilities)

open data city hackathon – look up more.

fascinating stuff. “getting health inspector data before you go into the restaurant”

scalability: breadth, a lot of people can do the same thing at once.

he’s more of a server-side guy, but don’t ignore front-end. (which was all in yesterday’s talk)

diagram of process of getting to the site. browser cache, if it’s accurate, is best of all possible worlds, speed-wise

caching at multiple levels: it’s caches all the way down.

a lot of f-bombs in this presentation. interesting.

history of drupal: first actually popular drupal site: the website fell down under slashdotting. so, very basic page caching, which is still with us.

PHP caching! dynamically compiled language, when you make a request, reads all of the files and compiles them into executable code on the fly. cool, but slow. “build an app every time you make a page request” so then OpCache, APC, speeds site by “like 10x” (should probably check on that.)

CSS/JS aggregation, starting with D5. 3 requests rather than 35.

Pressflow? Friendly fork of drupal made for high performance. Can generally install right over existing drupal installation. huh. http://pressflow.org/

Big sites don’t upgrade until contrib catches up, and until big projects upgrade, don’t always know where the pain points are…and then drupal is frozen, feature-wise.

Link back to the xhprof session for finding the spots where things break down. (damn I wish I’d had that during the themekey debacle.)

not every drupal problem has a drupal solution.

(yesterday I was too warm. today I’m too cold. dammit.)

Varnish is so much better than page cache for slashdotting-type issues. (anon visitors hitting one page all at once) Still invoking PHP, querying database, etc. https://www.varnish-cache.org/

because it’s smart, it’s slow. where Varnish is stupid/simple, but really really fast.

mysql: the end of all optimizations – if you’re stuck optimizing that, then you’re at the last resort.

downside of drupal caching: still hitting the database.

memcache abstracts that layer (oh, hey, didn’t this come up earlier, in some other session?) – improvements to scale (vs speed) – take the busywork away from the smart systems.

search! “mysql/php possibly the worst option for full-text search” eep.

apache solr as the right away to do search, because it was built from the group up to do that. built-in porter-stemmer. whoa.

stuff for the future. views with solr?

takes a lot of CPU cycles to process Drupal page requests, esp as you add modules.

no more special snowflakes.

stuff pantheon is doing…includes a site for the UN.

the one-click dream.

“offcache”?

boost module – big step forward.

PHP stream wrappers? public:// private:// temporary:// – interesting. I had no idea.

dynamic image modification now in core, along with file and image fields.

adding fields to files.

read-only stream wrappers – ie, youtube. is that the sort of thing I’m looking for?

Palantir getting involved, notable client American Public Media. (would imagine they have similar use cases?)

philosophy: reuse/reduce/recycle – rather than writing custom stuff.

two branches: one stable, the other with shiny new stuff

the Future!

I’m wondering: what does awesome look like?

looking at module page: “Flexible display filters for image manipulation, audio & video players and more.” – that’s what I’m looking for. Excellent, now I have the right option for one of my “try alts” entries in my spreadsheet.

needs moar screenshots.

making a commitment to maintain this module. “if I choose this module, what kind of future does it have?” indeed.

want to make sure, though, that other people can get involved, that it’s not just this one company’s project.

media core initiative?

free ticket to drupalcon denver for volunteering on sprint?!

oh yay demo!

select media button – modal window does look like wordpress. “view library” is actually a View. could use View pager infinite scroll (?) plugin.

looks like the process is still getting ironed out, altho apparently there’s a default view that ships enabled.

guy in the audience is using 2.x – “I like spicy foods” – and it works really well for him.

wysiwyg integration – “kinda needs a lot of work”

what about other submodules staying in sync with 2.x (ie, youtube, vimeo, etc)? relying on maintainers of those modules. still not stable, so not actually recommending those integrated submodules. “oembed”

audio player? not much lives in media right now. ok, so maybe not so much. he recommends “media element” module: http://drupal.org/project/mediaelement html5, even!

but I do like how

can you replace the image and keep the fields? media replace module – works with media 1.x – something they want to consider putting into the main module.

Community conversation: burnout, what can the community do help people avoid burnout.

A well-studied clinical condition, not a fad! Important scholarly work done: exhaustion, cynicism, ineffectiveness. (o hai n’hood assoc!)

causes: feeling overloaded, lacking control over what we do, not being rewarded, breakdown in community, not being treated fairly, dealing with conflicting values.

rewards: giving attribution, pats on the back…but easy to find people who feel ignored (which is the open source version of reward?)

“family problems” – at tense times, “it’s all going to hell” — and it didn’t. making personal statements on twitter, inflamatory comments on drupal.org, etc.

lack of boundaries with professional projects vs hobbyist.

slide of 12 stages of burnout.

individual solutions.

“real interconnectedness in a local community” which is hilarious for me, because that’s where I’m actually experiencing burnout. I think my 4e (D&D) game is my alternative/outlet for that.

burnout is NOT an individual problem, but an organizational problem.

specific things that Drupal does that make things work

calls out Dave Reid – has created 87 modules – module creators automatically become module maintainers. Even Dave has a limit. what do we do about that problem?

no explicit way to step down from a lot of responsibilities: no natural way, just guilt and overwhelmedness. “abandoned module hopper” -

“what’s wrong with assigning to abandoned”

“adopt-a-module”

not just name change – realizing that we have to promote people taking over and giving up responsibilities.

“there’s a process for adopting humans” (audience comment)

figuring out how to mentor and helping people to grow

you have to step away for people to feel comfortable stepping in

audience member – traveling is a universal way to “break the chain of misery” – also, finding a good psychologist. It really was my therapist who helped me work through that I needed to step away from friends of the library and recognize that I felt better after having done it. (because I’ve been going through whole life burnout, really)

tarpits – things that are not sustainable and will never work – not just module maintainership – docs team, good steps towards sustainability – project application process. sometimes you just can’t work harder.

organizational development – audience member’s wife does that professionally. no position for a documentation lead, for example, on modules. interesting idea: not having to be a co-maintainer to contribute to a module.

things that we do right!

don’t forget the risks for not trusting newbies.

time-limited responsibilities – reminds me of the n’hood association board – knowing that I have a term, and that it has an end point, is really helpful.

dude, I think greggles just trolled randy’s definition of bikeshedding. hilarity.

no way to resolve disagreements, is that what he’s talking about? to some extent.

I had an Excel spreadsheet — or rather, two spreadsheets — with the names of the locations, street addresses, application ID numbers, date of inspection and a few other bits. I imported them with Node Import, which was really easy, and tried a bunch of stuff to get the locations plotted on a map. Finally got it tonight! With this tutorial: http://drupal.org/node/1041632 – although the critical bit is that you have to set up a geocoding service before anything will work.

It’s pretty late right now, but I’ll try to go into a bit more detail about the process later.

usually don’t have to start from scratch, even if goal is to do something complex! often adding to or building on existing module.

will encourage you not to write a module if you don’t have to.

hooks – extend existing action.

I’m performing a particular action – does anyone have anything to say about it? I’m getting up to get napkins, does anybody want anything else? Yes, please get straws. I’m presenting a form to the user – please add a checkbox to that form.

hook_form_alter

hook_user

hook_nodeapi

hook_mail

form API – an array that translates to html. I like arrays. includes some automatic security protection.

sample project American Society for Delicious Foods (ASDF) – redirect user to appropriate category page after submitting a recipe.

hook_form_alter & dpm (a Devel function) – easily print any variable to the screen, instead of printr()!

kitty!

.info – tells Drupal that module exists – .module – contains main module code.

[note that I think all this is all in D7, but apparently concepts are the same]

find out about the node that was submitted – in Devel tab, find the

API site – looking up hook_form_alter – copy/paste function name & options

replace “hook” with the module’s name

devel will show a list of all the form IDs on the page, which you can use in the code.

really nice that he paused and asked if anybody was lost…and a few people were lost, and he’s being really good about addressing them.

using the dpm() function in the new module to print out information about the form.

find the save button in the form array. node_form_submit – that’s the function that gets called. add a second function after that one.

dpm($form['actions']['submit'], etc) – and then just see the part of the form that you want to understand.

but how to know what to do? – find example of how to do stuff when submitting a form.

when you create a function that does something, you add it to the array.

group debugging! possibly a typo.

shows that another function exists, and then the function runs at that point in the process.

form state -> values -> field_type_recipe, etc., etc.

interesting gotcha re: PHP constant LANGUAGE_NONE (?)

form_state[redirect] and set the path, using the term id that was extracted.

yay, it worked!

that’s pretty cool, actually.

helpful: variable_get, variable_set, user_access (related to permissions), t() – for translating interface text, makes it possible for others to translate your module,

common pattern: implement more or one hooks, use drupal API helper function, then do something custom.

func_get_args – see what arguments are passed to a function; helpful to compare what args get passed to successful operation vs unsuccessful one.

debug_backtrace – useful for WTF moments.

ezrabg on twitter.

I might be able to rewrite my multisearch module and even know WTF I’m doing.

example module!

maybe now I have a hope of adopting Ads Views.

particular page that’s running slow – exclusive

I don’t understand what’s going on.

80ms content type info – why is it being called so many times – look at time spent vs how many times it was called.

this is like the opposite of the last session – no pretty slides, lots and lots of examples. equally baffling, tho.

query logger

take a look at memcache.

(it occurs to me that I’m going to want to take a big chunk of time this fall to really clean up the site.)

likes to see mysql problems float to the top because they’re easier to deal with than other problems.

can see exactly which queries are the problem.

this is all about how to identify problems, not how to solve them, which is fine. worst thing for me is just being baffled by why there’s even a problem.

SO FRUSTRATED that I missed how to get this up and running.

Devel query logger?

Google “[function name] performance” and look for patches, etc.

Ok, so XHProf is an extension for PHP. what’s going to be involved in getting that up and running?

am I caching Views correctly?

all about finding low-hanging fruit and/or biggest bottleneck.

workarounds vs

xdebug – what did he just say was the gui? but xhprof is easier, includes some features

devel query log, views signature – what else does this give you? module problems.

APC? “this are the things that you have already done before this can be useful” – missed some details in his answer to the question that I didn’t actually hear very well. (from webchick?)

so maybe I should look at devel query log FIRST, but this is probably useful too.

is he talking about the xhprof module? or xhprof in devel? xhprof module is coming back to life, I guess; makes more sense than devel integration.

I can’t hear any of the questions, especially since the VACUUM CLEANER IS RUNNING. (so I shut the door.)

q: was originally designed for evaluating a single page – how is this useful for a whole site? that’s why he’s doing work on extending xhprof for drupal. or something. I am genuinely baffled. (once again, sorta wishing I’d gone to a different session. Omega theme might have been handy.)

CPU time and function calls are key info.

(seen on Twitter: Omega has a subtheme wizard! session info at http://omega.cellardoormediagroup.com/)

slow query logger.

threshhold of a good CMS is how easy you can make variables pretty. template is an abstraction of a static html page.

“felt like I could never get the granularity that I wanted”

D7 gives more control.

he maintains Fusion module for top notch themes.

1st impression of render api (D7) was negative. feels like stuff output by render is wrappers around wrappers around wrappers – “I can’t scroll that far to the right”

#theme thing in content array gives bare html? may have missed something.

content array – each field is templated according to a theming function -

I really want to see some code examples.

render api provides some new hooks – hook_page_build()

I’m missing which aspects are for D6 and which are for D7.

He has a great voice, but I’m having a hard time getting anything actually usable.

Things that sound hard can be really easy (with Views!) and things that sound easy can be really hard (adding a thing to a page).

hook_page_build()

Wow, some of that sounds cool. I WANT TO SEE IT ON THE SCREEN.

“29 hours of travel” – that may be why this presentation feels so disjointed.

having a lot of template files can really slow down site. (eep)

Well, this does make me curious about checking out theming in D7, so that’s something.

advanced aggreggator – advagg – supposed to help with the disappearing CSS! – http://drupal.org/project/advagg

D7 projects: agrcache, core_library

“don’t know if you’ve ever been woken up because your site didn’t look right” – eep. At least I’ve never had that.

the air conditioning in this room IS REALLY LOUD.

problems with not knowing if cache has cleared or not.

gzip, minification, longer browser cache lifetime (sounds like some weird conflict issues between Drupal cache/CSS filename rewriting and browser cache), use google’s cdn for jquery (can that work with ssl?), smart bundling – if you do one change, it’ll only push out new files related to that change – 404 css/js protection – I’m pretty sure that’s a big part of the vanishing CSS issue.

(I wonder if this will help with the vanishing menus.)

css embedded images module – reduces http requests – up to 24k of images (IE restrictions!), so good for tiny layout bits & pieces

Oh, and a bit of quick Googling turns up that the jquery CDN will work over SSL.

cdn module, cookieless_subdomain – so not sending cookies when sending images, makes easier to have parallel domains? either/or: NOT BOTH.

(could/would we run our own cdn?)

files proxy module, file conveyor – not sure if it would be useful in my situation.

changes to .htaccess to extend expiration – “access plus 1 year” – images, css, js – some discussion about query strings and cdns, an issue thread in cdn module queue.

gzip – can be done at php level (inside Drupal) or at apache. sample rules.

if mod_expires, disable etags, if not, enable etags.

putting scripts at the bottom, better to use a module than to do it in the theme: headjs (6 only), labjs (6 and 7?). but some issues with panels, wysiwyg editors. can trim about a second off time. we could probably use it; I’ve done some editing inside panels, but it’s really really rare. headjs requires advagg.

optimizing images – although it sounds like that’s an issue with images uploaded and processed with imagemagick. but might be worth looking into. patch for imageapi. also imageinfo_cache.

advagg_add_css_inline() – not sure about use case for that.

split components across domains, with cdn module. yep, that’s exactly what I thinking that would be. (hilariously, this harks back to an argument C & I got into years and years ago…in which I was completely wrong. thanks, Shelley! but it was good to understand better.)

jquery plugin lazyload.

don’t scale images in html. not generally a problem for me, but might be good for intranet; image resize filter.

favicon in core is 400% bigger than needed. ::facepalm::

webpagetest.org

showslow.com – every 24 hours, get yslow score, see over time.

google page speed – which I saw in the analytics training I was at earlier this week!

some sprite recommendations – I’m not sure if we have anything that would be good with that.

inline image tags – so, wait, printing the binary code inside the html? – particularly for mobile – although that seems like the opposite of the cdn recommendation.

slides: bit.ly/oRdwbp