6098 comments
2357 subscribers
6263 on Twitter
Subscribe! Feed reader E-mail

Drupal in the trenches: MySQL and DELETE joins; multiple tests

Today was the first time I used a DELETE statement that joined multiple tables. I didn’t even know you could do that. =)

I needed to look into this because my update function was just taking so much time and memory. We have a little over 3,000 nodes, most of which are usernodes automatically created by the usernode module. I needed to rebuild the node access entries for a few hundred content nodes to take into account the new partner access module I wrote, but node_access_rebuild took quite a while.

The first approach I thought of taking was to delete usernode because we didn’t seem to need it any more. In order to do that, though, I would still have to delete all the user nodes, though, and invoking all the hooks for each of the 2000+ user nodes took a while as well.

The second approach was to delete just the node access records related to the nodes I needed to update. I found a way to do it even faster – instead of querying the node IDs and then deleting individual rows, I could do something like this instead:

db_query("DELETE FROM na USING {node} n
          INNER JOIN {node_access} na ON (n.nid=na.nid) WHERE n.type='%s'", $type);
$rs = db_query("SELECT nid FROM {node} WHERE type='%s'", $type);
while ($row = db_fetch_array($rs)) {
  $node = node_load($row['nid'], NULL, TRUE);
  if (!empty($node)) {
    node_access_acquire_grants($node);
  }
}

(UPDATE: Fixed MySQL statement)

That was interesting. =)

I was also running into memory problems trying to run all the unit tests in one go. I broke it into individual tests with:

drush test list | grep KEYWORD | xargs -n 1 drush test run | tee out

where KEYWORD is the keyword I use to filter the list for just the tests I wanted to run, and out is the file that stores a copy of the results. It’s not neatly summarized, but oh well. =) It works.

Short URL: http://sachachua.com/blog/p/6319
  • http://synapsefire.blogspot.com/ Stefan Kudwien

    Beware though, this is a MySQL-specific extension to DELETE. FWIW, an SQL standards compliant approach would be using a subquery:

    db_query("DELETE FROM {node_access} WHERE nid IN (SELECT nid FROM {node} WHERE type = '%s')", $type);

    However, this will most likely be slower.

  • http://robshouse.net Robert Douglass

    I ran into a nearly identical problem yesterday with an upgrade script that needed to delete 6000 nodes. It seems to me that, in Drupal 7, now that we track foreign keys, we should be able to get rid of a lot of the deleting that happens through hooks and just depend on cascading deletes at the database level. Hopefully this will speed the mass deleting of nodes even further. I believe that Drupal 7 also allows for sending in an array of nids for deletion, but haven’t tested to see if it helps the case of mass deletion significantly.

    Here’s the commit that puts foreign keys into D7.
    http://is.gd/M4Dy

On This Day...

  • 2012: The Shy Connector: Thinking about the difficulties people encounter — I want to revise and expand on The Shy Connector, which seems to have become a perennial resource. I was [...]
  • 2011: Hungry hungry compost and other garden updates — The compost bin chomps through all the organic material we give it. It’s nowhere near the smooth dark brown of [...]
  • 2011: VMWare, Samba, Eclipse, and XDebug: Mixing a virtual Linux environment with a Microsoft Windows development environment — I’m starting the second phase of a Drupal development project, which means I get to write about all sorts of [...]
  • 2010: Weekly review: week ending May 30, 2010 — From last week’s plans: Work [X] Talk to people about expertise location and other new tools [X] Talk to [...]
  • 2008: Drupal: Lather, rinse, repeat – Cleaner development with installation profiles and Makefiles — I’m beginning to really like Drupal, a PHP-based content management system. Working with Drupal feels almost like working with Emacs [...]
  • 2008: Weekly review: Week ending June 1, 2008 — Last week: Gave two presentations in person: I.B.Millennials at the Regional Technical Exchange at Sheraton Parkway, and Top 10 [IBM] Web [...]
  • 2007: Global opportunities in software development: talk by Winston Damarillo: 21:37 — I attended Winston Damarillo’s lecture on global opportunities in software development today. The audience included faculty from UP, Ateneo, and DLSU, three [...]
  • 2007: Party — We made it back to Manila in time for the party for IT geeks at the Museum Cafe in Greenbelt. The [...]
  • 2006: Wow, what a blogger! — Greet this blogger happy 83rd birthday, folks! Way, way cool. Link from Sandy Kemsley, Helen K’s daughter. On Technorati: blogging
  • 2006: Impromptu barbecue — James Iveniuk reminded me that we’d planned to have some kind of barbecue tonight. I invited Jed Smith as well because [...]
  • 2006: Interesting people, interesting conversations — It is my firm belief that if I mash interesting people together, they’ll most probably have interesting conversations – and if [...]
  • 2006: Blog hiccup — Sorry about the blog hiccup. I messed up my template last night. =) Random Japanese sentence: 鬼の居ぬ間に洗濯。 When the cat is away, [...]
  • 2005: Wahoo! iPod Photo — So much for not being a gadget freak. =) Dad got me an iPod Photo (30GB). After I formatted it to [...]
  • 2004: Testing Japanese — テスト。テスト。私はチュアです。
  • 2004: Task lists — If we store all tasks in a plain text file, we can generate task lists based on that text file instead. [...]
  • 2003: Funny Linus quote — Seriously, the way I did this was by using a special /sbin/loader binary with debugging hooks that I made (“dd” is [...]
  • 2003: More head-mounted displays — Jennifer Pellinen, wear-hard: I found this HMD that looks pretty decent and its a decent price. http://www.vrealities.com/vr1.html
  • 2003: Added TeachingReflections — Check out my TeachingReflections – stuff I learned from CS21A.
  • 2003: More problems in computer science — this website might be useful http://chortle.ccsu.ctstateu.edu/cs151/cs151java.html
  • 2003: More funky keyboards — - http://www.extremetech.com/article2/0,3973,1034116,00.asp - http://www.fingerworks.com/touchstream_products.html - http://www.cdc.gov/niosh/pdfs/97-148.pdf
  • 2003: Funny disclaimers — http://www.they.com/disclaimer/
  • 2003: OSS advocacy stats — http://www.dwheeler.com/oss_fs_why.html
  • 2003: More articles about women — “Making It on Their Merits” The U.S. high-tech sector is still predominantly male, but women have made significant progress in attaining prominence [...]

Get the highlights as a PDF!

Stories from my Twenties: Highlights from a Decade of Blogging

Free sample!