Deck 07: HDFS

Deck 07: HDFS

Q1. HDFS has a boss/worker architecture.  What are the specific names given to the boss and workers in an HDFS cluster?

Q2. When a client wants to read an HDFS file, how does it interact with the NameNode and DataNodes?

Q3. What are pipelined writes?

Q4. If you write 1 MB to a triply replicated file, how much disk I/O does that generate?  What if you read 1 MB from a triply replicated file?

Q5. How do you format an HDFS NameNode?

Q6. How do you start running an HDFS NameNode with hostname “main” and port 9000?

Q7. How can you get an HDFS cluster report for hdfs://main:9000/?

Q8. How can you start a DataNode and connect it to a NameNode with hostname “main” on port 9000?

Q9. If you want to use commands like mkdir, cp, ls, cat, and du on HDFS, how do you do it?

Q10. The HDFS file system was modeled on a paper about the GFS file system at Google.  What is the newer distributed file system Google built and started using after GFS?

Q11. What does GFS stand for?

Q12. What is the Hadoop system similar to GFS?

Q13. What is a Hadoop system similar to Google’s BigTable?

Q14. What is a distributed analytics system that many people have started using instead of MapReduce?

Q15. When Google MapReduce and BigTable were first created, where did they store their data?

Q16. What is the technical word for having multiple copies of our data on different systems?

Q17. What is the technical word for breaking our data (for example, a file) into smaller pieces that are more manageable?

Q18. What is a weakness of HDFS’s approach to partitioning?

Q19. What is the open source system most similar to Google’s BigTable?

Q20. How do HBase and Cassandra handle replication differently for N-times replicated data?

Q21. HBase and BigTable are “versioned sparse tables” — what does this mean?

Q22. What happens to the regions an HBase RegionServer was responsible for if that server dies?

Q23. Why might you want all data for a particular user to live in a single row in an HBase table?