9/12/2016

Hadoop in real world (UDEMY online course)

In taking the Hadoop in real world online course at udemy.com, I ran into a couple of issues / items to work through in order to be successful with several of the lectures.

For starters the course relies on using Eclipse, JAVA, and files provided by the course.  The lectures in Section 1 walk through the tools you're going to need.  In order to proceed with the course I installed the following software:

WinSCP-5.9-Portable
  • I used this to download the hirw-workshop folder from the lab server
Eclipse Java EE IDE
  • The use of codenames can make things very confusing for first time users
    There's another breakdown here.
  • I managed to end up with the Kepler version.
    (If you pay attention in lecture 2 you'll see the instructor uses the "mars" version.)
I had trouble getting the eclipse packages to perform the maven build successfully.  This happened for multiple reasons.

The first issue I ran into was that I did not yet have JAVA installed.  The JAVA selection of versions is again a veritable quandary.  Do you want SE, EE, ME?  ...then the JDK or JRE version?
I went with Java SE Development Kit 8 (JDK).

Once that was setup I needed remove the comment from the pom.xml files to include the jdk.tools. However, the install of JAVA did not create the environment variables so I had to set them up manually:

How to setup the Java environment variables:
Control Panel\All Control Panel Items\System > Advanced system settings

Advanced > Environment Variables

Under System variables > New
  • JAVA_HOME - C:\Program Files\Java\jdk1.8.0_101
  • JRE_HOME - C:\Program Files\Java\jre1.8.0_101
Once Java was able to be successfully referenced, I was still having build errors:
"Failed to execute goal org.apache.maven.plugins"
Stackoverflow.com to the rescue!

I resolved this by updating the Eclipse JRE reference (to the JDK instead):
Run > Run Configurations...
JRE Tab, click "Installed JREs..." next to Alternate JRE:
(mine shows "jdk1.8.0_101, which is what I set it to (I.E. it's fixed in the screenshot below).
From here we click Add:
Next

Click Directory and brows to the JDK, mine was here:
C:\Program Files\Java\jdk1.8.0_101

Once that was set to the JDK I unchecked the JRE reference under the "Installed JREs" and clicked OK, then close.

I was then able to successfully complete Run as...>Maven build.

After setting this all up I noticed the instructor was using the following:
Java SE Development Kit 7
Eclipse Mars

2 comments:

  1. please attach a link of hirw directory

    ReplyDelete
    Replies
    1. It was a while ago, they emailed me the link when I purchased the course. It could be different now.

      Delete