Skip to content

Naiad at the GraphLab workshop

by on July 1, 2013

I am going to be presenting presented our work on Naiad at the GraphLab workshop, this afternoon at 3:10pm. I’ll talk about the trade-offs between data-parallel and graph-parallel systems, and show how Naiad delivers the best of both worlds.

You can download the slides. The PageRank numbers that I presented were obtained using a data layout based on Hilbert Space-Filling Curves, and Frank has a separate post with details of this approach.

We encourage you to check out the project web page, our blog posts about Naiad, and the code release. If you have any questions, leave a comment or email us: naiadquestions at microsoft dot com.

From → Naiad

2 Comments
  1. Abhishek permalink

    hi, I was trying to see if Naiad could be of use for our graph data. I loaded a test data set of 10M nodes and edges on my windows workstation and was impressed with the speed of results but when I bumped up the node and link count to 1 B, it crashes with an exception.

    Mine is a windows 7 8 core 24 G RAM workstation.

    I can provide the stack trace if you want but I’m pretty sure its nothing unusual from what you have seen. Wondering if there is a limit to the node and edge count that Naiad can handle as of now?

    • Hi Abhishek,

      There are definitely limits on how much can be practically processed, but the reasons for the limits vary a bit. By default .NET isn’t happy with any object larger than 2GB, for example, and it could be that the 1B long array of edges tripped that. If you send a stack trace to naiadquestions at microsoft dot com, we can try and sort out what is causing the exception.

      Cheers,
      Frank

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 62 other followers

%d bloggers like this: