AgilePHP » Hadoop http://www.agilephp.net 开发可以敏捷，人生需要慢慢体味 Thu, 03 Nov 2011 11:53:07 +0000 en hourly 1 http://wordpress.org/?v=3.3.1 折腾Hadoop的一天 http://www.agilephp.net/2009/09/a-day-z-turn-with-hadoop/ http://www.agilephp.net/2009/09/a-day-z-turn-with-hadoop/#comments Mon, 21 Sep 2009 12:52:26 +0000 bbiao http://www.agilephp.net/?p=174 相关文章
  1. Apple iPad
  2. Ubuntu下配置Web环境
  3. 了解一些“云”
]]> Hadoop自己的文档写的也确实不是特别清楚，抛去折腾（英文：z-turn）cygwin的时间，也花了我将近两个小时才把Hadoop Cluster配置起来。主要参考了两篇文档：

Cluster Setup

Running Hadoop On Ubuntu Linux (Multi-Node Cluster)

这里总结一下我的经验，把hosts各主机的主机名配好（生产中的应用应该用name server来作），然后在某台充当NameNode和JobTracker的机器上把Hadoop配置完成以后，将该配置好的Hadoop通过rsync的方式同步到其他机器上去，然后修改NameNode和JobTracker里的masters和slaves文件，即可。

之所以没在linux下做实验，是因为有些任务需要在windows下跑，所以才z-turn了一下cygwin。明天要z-turn一下在Hadoop上写自己的MapReduce程序了。

相关文章

  1. Apple iPad
  2. Ubuntu下配置Web环境
  3. 了解一些“云”
]]> http://www.agilephp.net/2009/09/a-day-z-turn-with-hadoop/feed/ 1 了解一些“云” http://www.agilephp.net/2009/08/cloud-to-overview/ http://www.agilephp.net/2009/08/cloud-to-overview/#comments Mon, 24 Aug 2009 09:11:11 +0000 bbiao http://www.agilephp.net/?p=148 相关文章
  1. 折腾Hadoop的一天
  2. 条件注释
  3. Portable Firefox中安装Flash插件
]]> Windows Azure Platform

The Windows Azure Platform provides a wide range of internet services that can be consumed from both on-premises environments or the internet.

Apache Hadoop

The Apache Hadoop project develops open-source software for reliable, scalable, distributed computing. Hadoop includes these subprojects:

  • Hadoop Common: The common utilities that support the other Hadoop subprojects.
  • Avro: A data serialization system that provides dynamic integration with scripting languages.
  • Chukwa: A data collection system for managing large distributed systems.
  • HBase: A scalable, distributed database that supports structured data storage for large tables.
  • HDFS: A distributed file system that provides high throughput access to application data.
  • Hive: A data warehouse infrastructure that provides data summarization and ad hoc querying.
  • MapReduce: A software framework for distributed processing of large data sets on compute clusters.
  • Pig: A high-level data-flow language and execution framework for parallel computation.
  • ZooKeeper: A high-performance coordination service for distributed applications.

IBM HiPODS

HiPODS has evolved to developing innovative, first-of-a-kind solutions to deeply challenging on demand business issues in such areas as:

  • Web 2.0 technologies
  • Collaborative innovation solutions
  • Cloud computing
  • Automated IT services management
  • Capacity planning for Service Oriented Architecture (SOA) systems
  • Software as a service

相关文章

  1. 折腾Hadoop的一天
  2. 条件注释
  3. Portable Firefox中安装Flash插件
]]> http://www.agilephp.net/2009/08/cloud-to-overview/feed/ 2