You know we can Hadoop runs in 3 modes.
1.Local (Standalone) Mode
2.Pseudo-Distributed Mode
3.Fully-Distributed Mode
Local Mode: By default, Hadoop is configured to run in a non-distributed mode, as a single Java process. This is useful for debugging. (single machine)
Pseudo-Distributed Mode: Hadoop can also be run on a single-node in a pseudo-distributed mode where each Hadoop daemon runs in a separate Java process. (single machine)
Fully-Distributed Mode: Here Hadoop runs on multiple-nodes in a distributed mode.
We will use Pseudo-Distributed Mode for practice.
Cloudera is one of the leading vendors of Hadoop, provides Cloudera vmware where they've installed all required components of Hadoop like HDFS,YARN,Hive,Pig,Oozie,Sqoop,Flume etc. Starts all these services when you start Cloudera VM. So you can concentrate on programming.
In order to execute Cloudera Hadoop CDH4 VM in Windows , you need to download the quickstart VM from here according to your VM version(i.e VMware/VirtualBox/KVM). It requires a 64 bit host OS.
This VM runs CentOS 6.2 and includes CDH4.3, Cloudera Manager 4.6, Cloudera Impala 1.0.1 and Cloudera Search .9 Beta.
I have used VMware version of Cloudera Quickstart VM for running on Windows 10 64 bit host OS.
System Requirements:
This is a 64-bit VM, and requires a 64-bit host OS and a virtualization product that can support a
64-bit guest OS.
This VM uses 4 GB of total RAM. The total system memory required varies depending on the size
of your data set and on the other processes that are running.
The demo VM file is approximately 2 GB.
To use the VMware VM, you must use a player compatible with WorkStation 8.x or higher: Player 4.x or higher, ESXi 5.x or higher, or Fusion 4.x or higher. Older versions of WorkStation can be used to create a new VM using the same virtual disk (VMDK file), but some features in VMware Tools won’t be available. You can download vmware workstation player from here.
After downloading the Cloudera VM , extract it & select the virtual machine configuration (.vmx) file into your vmware player.
Start your vm player and click on Open a Virtual Machine as shown in above and navigate to extracted Cloudera vmware folder and select cloudera-quickstart-vm-5.7.0-0-vmware.vmx file.
after selecting the above file on left nav you can see the cloudera quick start vm. By clicking once you can edit the player to edit RAM (min 4GB you've to keep otherwise it crashes).
By double clicking on cloudera quick strat vm you can start cloudera.
Once cloudera starts it may ask you credentials cloudera/cloudera is username/password.
Open terminal (marked in yellow) and start practising Unix Commands and HDFS commands
Previous Post Next Post
1.Local (Standalone) Mode
2.Pseudo-Distributed Mode
3.Fully-Distributed Mode
Local Mode: By default, Hadoop is configured to run in a non-distributed mode, as a single Java process. This is useful for debugging. (single machine)
Pseudo-Distributed Mode: Hadoop can also be run on a single-node in a pseudo-distributed mode where each Hadoop daemon runs in a separate Java process. (single machine)
Fully-Distributed Mode: Here Hadoop runs on multiple-nodes in a distributed mode.
We will use Pseudo-Distributed Mode for practice.
Cloudera is one of the leading vendors of Hadoop, provides Cloudera vmware where they've installed all required components of Hadoop like HDFS,YARN,Hive,Pig,Oozie,Sqoop,Flume etc. Starts all these services when you start Cloudera VM. So you can concentrate on programming.
In order to execute Cloudera Hadoop CDH4 VM in Windows , you need to download the quickstart VM from here according to your VM version(i.e VMware/VirtualBox/KVM). It requires a 64 bit host OS.
This VM runs CentOS 6.2 and includes CDH4.3, Cloudera Manager 4.6, Cloudera Impala 1.0.1 and Cloudera Search .9 Beta.
I have used VMware version of Cloudera Quickstart VM for running on Windows 10 64 bit host OS.
System Requirements:
This is a 64-bit VM, and requires a 64-bit host OS and a virtualization product that can support a
64-bit guest OS.
This VM uses 4 GB of total RAM. The total system memory required varies depending on the size
of your data set and on the other processes that are running.
The demo VM file is approximately 2 GB.
To use the VMware VM, you must use a player compatible with WorkStation 8.x or higher: Player 4.x or higher, ESXi 5.x or higher, or Fusion 4.x or higher. Older versions of WorkStation can be used to create a new VM using the same virtual disk (VMDK file), but some features in VMware Tools won’t be available. You can download vmware workstation player from here.
After downloading the Cloudera VM , extract it & select the virtual machine configuration (.vmx) file into your vmware player.
after selecting the above file on left nav you can see the cloudera quick start vm. By clicking once you can edit the player to edit RAM (min 4GB you've to keep otherwise it crashes).
By double clicking on cloudera quick strat vm you can start cloudera.
Once cloudera starts it may ask you credentials cloudera/cloudera is username/password.
Open terminal (marked in yellow) and start practising Unix Commands and HDFS commands
Previous Post Next Post
No comments:
Post a Comment