Django – Hello World

•February 17, 2017 • Leave a Comment

Wow, this was pretty easy (especially compared to Ruby on Rails). But then, Python and pip were already installed. So to get the binary, I simply

sudo pip install Django

I made a special user to run the server

sudo useradd django -d /home/django -M -p django
sudo cp -R /home/ec2-user /home/django
sudo chown -R django:django /home/django

and created a folder to work in

sudo mkdir -m 777 /Django

and a new project, using the newly installed binary

django-admin startproject Introduction

now I can start using this tutorial. It talks a lot more in-depth about everything. I stripped it down to the essentials. I started the server

python manage.py runserver 0.0.0.0:8000

0.0.0.0:8000 is necessary to make the page accessible from an external machine. But still, this kicked out a DisallowedHost Invalid HTTP_HOST header error, so edit /Django/Introduction/Introduction/settings.py and add the server IP to the ALLOWED_HOSTS line in /Django/Introduction/settings.py

ALLOWED_HOSTS = ['10.4.0.1']

now I visit 10.4.0.1:8000, and it works

it_works

Now, create an app to house the actual code

python manage.py startapp hello_world

create a view. This is what Ruby on Rails calls a controller (also, Ruby on Rails is to view what Django is to template). Edit hello_world/views.py, and replace the contents with

from django.http import HttpResponse

def index(request):
    return HttpResponse("Hello, World.")

then replace the contents of hello_world/urls.py with

from django.conf.urls import include, url
from hello_world import views

urlpatterns = [
    url(r'^Introduction/hello_world$', views.index, name='index'),
]

I ran the server, headed to http://10.4.0.1:8000/Introduction/hello_world, and…

hello_world

Ruby – Hello World part 2

•February 10, 2017 • Leave a Comment

Well, I couldn’t turn my brain off from solving this problem. I decided to try installing Ruby and Rails from source in their own folder structure. Therefore, everything that should be needed should simply get compiled and all reside in the same place, right? Well, nearly.

I used the high level steps from this guide with the url for the latest version of Ruby from the downloads section of ruby-lang.org

Although I had OpenSSL installed, I needed the compilation hooks for Ruby and Gems to work right

sudo yum install -y openssl-devel

If I’d started this process fresh, I might have needed more than that. I cleaned every trace I could find of Ruby and gems, so I THINK I was starting fresh.

I downloaded the Ruby source and expanded it.

sudo mkdir -m 777 /RoR
cd /RoR
wget https://cache.ruby-lang.org/pub/ruby/2.4/ruby-2.4.0.tar.gz tar xvzf ruby-2.4.0.tar.gz
tar xvzf ruby-2.4.0.tar.gz

now, I made a folder, and compiled everything into it, just so everything was quarantined

mkdir /RoR/build
cd /RoR/ruby-2.4.0
./configure --prefix=/RoR/build
make
make install

note there isn’t any sudo’s in there. I’m installing to a folder I made, so root doesn’t have any part in this. Now since OpenSSL hooks were installed, I can

cd /RoR/build/bin
./gem update
./gem update --system

I saw json updates go by. I already know I’m doing better than before.

now install rails

./gem install rails

I saw nokogiri 1.7 get compiled without a big to-go. I should have done it this way the first time!

now create my hello world app

cd /RoR
mkdir Apps
cd Apps
/RoR/build/bin/rails new hello_world

then try and start the server

cd hello_world
/RoR/build/bin/rails server

oh look, our old friend Error. I got rid of the first one with this

/RoR/build/bin/gem uninstall openssl -v 2.0.3
/RoR/build/bin/gem install openssl -v 2.0.2

and the second one (about a missing JavaScript interpreter) with the following (which was hard to find since it wasn’t in the main repo)

sudo yum install nodejs npm --enablerepo=epel

and now it runs, yay!

cd hello_world
/RoR/build/bin/rails s

go to http://10.4.0.1:3000 across my vpn and…youre_on_rails

and then a few more tweaks to get hello world. I followed this article, starting at step 4.2 to get started (it explains each step well, as well as the relationship of controllers to views). So, here we go, run

cd /RoR/Apps/hello_world
/RoR/build/bin/rails generate controller Welcome index

to create the controller.

Then, replace the contents of /RoR/Apps/hello_world/app/views/welcome/index.html.erb with

<h1>Hello, Rails!</h1>

add the line

root 'welcome#index'

to /RoR/Apps/hello_world/config/routes.rb

and now when you go to http://10.4.0.1:3000/:

hello_rails

I’m sure the ‘rails server’ command doesn’t constitute a “production server”, but I’m not worried about it right now. I’m just trying to learn Rails, not being a full Sys Admin.

Ruby – Hello World

•February 10, 2017 • Leave a Comment

update: I did get this to work, go to the next post. So anyone looking for instructions can safely ignore this post, because it ends in failure

Here we go! I’m going to try out running Ruby on my Amazon Web Services instance. First I login. Looks like Ruby is installed, but Rails would only work with Ruby 2.2.2 or newer. So, I ended up removing everything first

sudo yum remove ruby ruby20 ruby20-libs

all this may not be necessary, but I had installed a lot of stuff prior trying to get this to work. Run this

yum list installed | grep ruby

to see what you have installed, to decide what you may still need to remove. Now, install the correct version:

sudo yum install ruby22 ruby22-devel

gem comes with ruby, and devel is needed to let the subsequent gem updates succeed. Now, make sure gem is up to date

sudo gem update
sudo gem update --system

normally, the next step is to just tell gem to install Rails. But nokogiri had a problem installing, and then pkg-config was needed on top of that, as well as some libraries. So

sudo gem install pkg-config
sudo yum install libxml2-devel libxslt-devel

next, actually install nokogiri (with this hacky workaround. I hope at some point in the future they actually fix their installer)

sudo gem install nokogiri -- --use-system-libraries=true \
       --with-xml2-include=/usr/include/libxml2

and finally (finally) install rails

sudo gem install rails

which succeeded. But then, when I went to create my project

sudo mkdir -p /Ruby_Apps/Introduction
cd /Ruby_Apps/Introduction
rails new hello_world

I ran into errors with sqlite3, so I had to install the devel package

sudo yum install -y sqlite-devel

now it will work

rails new hello_world

then run the server (I chose to run it on port 8081)

cd hello_world
rails server -p 8081

/usr/local/share/ruby/gems/2.2/gems/activesupport-5.0.1/lib/active_support/core_ext/object/json.rb:2:in `require’: cannot load such file — json (LoadError)

oh no, another error! Quick, someone, anyone, tell me RoR is worth the trouble because it seems super flakey!

I tried lots of installing json, which didn’t ever seem to take. ‘bundle install’ from inside the hello_world directory, which didn’t seem to take, or maybe it didn’t know there was a problem. But my daughter is awake, and it’s been another week without much to show for it. So, I’m giving up for now. This guide seemed like a good high-level Hello World guide, except that Ruby broke at every step. Ah well, I might just work on Django as a pallet cleanser, and come back to this.

 

Hello Login

•January 27, 2017 • Leave a Comment

So I already covered how to write a class to encrypt and use an authentication file from authentication code. Let’s use that class for a simple example.

It took me a while to figure out that for Java, compiling is only the first step. The only compiled language that I have much experience with is C++. With that, when you reference packages and compile your project, everything ends up in the same binary. Java, however, seems to only compile the project byte code (C++ is to binary what Java is to byte code) with a reference to the package byte code. So I got my package where it needed to be to compile the project correctly, then moved the project byte code to the correct place for Tomcat, and everything broke! The other edge of this sword is that if you update and compile a dependent package, you don’t have to recompile everything

Also, where you run the javac command from matters. So if your package name is com.me.package, package.java needs to be inside the folder structure ‘com/me’. And the java file that will use it needs to be in the root, in this case, in the same folder as the ‘com’ folder. Finally, when running the javac command (with ‘.’ in your CLASSPATH. I updated my post with that, which prevents you from using -cp as in the second answer in this post), your working directory needs to be the root when compiling both files, such as the following:

javac com/me/package.java
javac project.java

both of these epiphanies would have come if I’d understood this article on the first read. As it turned out, it took me dozens of reads to understand what was needed.

So, with that out of the way, here’s what needs to happen:

My plan is simple:

  • When the page is first hit, prompt for a login.
  • If that login is wrong, send an error message and prompt for another login (don’t worry about keeping track of the number of attempts).
  • If the login is successful, and while the session hasn’t timed out, all further attempts to reach the page are met with the users “home” page.
  • After the session times out, return the login page (don’t worry about returning a “session has timed out”message).

So here we go:

// Import required java libraries
import java.io.*;
import javax.servlet.*;
import javax.servlet.http.*;

// bring in our custom package
import com.CB.PasswordEncryptionService;

// Extend HttpLogin class
public class HelloLogin extends HttpServlet {

    private PasswordEncryptionService pes;

    public void init() throws ServletException
    {
        // do nothing, but we need this because the servlet
        // container will call it
    }

    public void doGet(HttpServletRequest request,
                      HttpServletResponse response)
            throws ServletException, IOException
    {
        // get the session object. the 'true' creates a new
        // session if one doesn't exist
        HttpSession session = request.getSession(true);
        // set the session timeout length
        session.setMaxInactiveInterval(120);

        // Set response content type
        response.setContentType("text/html");

        // Actual response goes here.
        PrintWriter out = response.getWriter();

        // determin which type of page we need
        if (session.getAttribute("USER") == null) {
            // this happens when the USER is not set
            // which can happen when the site is hit for the
            // first time or a login was unsuccessful, or the
            // session has timed out
            out.println("<form method=\"post\">");
            out.print("<p>User Name: <input type=\"text\"  " +
                    "name=\"user\"");
            out.println(" size=\"40\"></p>");
            out.print("<p>Password: <input type=\"password\"  " +
                    "name=\"password\"");
            out.println(" size=\"40\"></p>");
            out.println("<input type=\"submit\" value=\"login\">");
            out.println("</form>");
        }
        else {
            // we have the users successful login
            String user = session.getAttribute("USER").toString();
            String salutation = "<h1>";
            if ((boolean)session.getAttribute("firstLogin")) {
                salutation += "Hello, ";
                session.setAttribute("firstLogin", false);
            }
            salutation += user + "</h1>";
            out.println(salutation);
        }
    }

    // I will only use he doPost for the actual authentication
    // step, then things will get handed back to doGet
    public void doPost(HttpServletRequest request,
                       HttpServletResponse response)
            throws ServletException, IOException {
        // Set response content type
        response.setContentType("text/html");

        // Actual response goes here.
        PrintWriter out = response.getWriter();

        // initialize the PasswordEncryptionService package
        pes = new PasswordEncryptionService();
        pes.readFile(
                getServletContext().getRealPath("/WEB-INF/users.db"));

        // retrieve variables
        String user = request.getParameter("user");
        String pass = request.getParameter("password");
        HttpSession session = request.getSession(true);

        // verify the user name and password...
        boolean valid = false;
        try {
            valid = pes.authenticate(user, pass);
        }
        // I am assuming this works. It compiles. The plan is for
        // my package to _not_ throw an exception
        catch (Throwable e) {
            e.printStackTrace(out);
        }

        if (valid) {
            // if this is a valid authentication, set the user ...
            session.setAttribute("USER", user);
            session.setAttribute("firstLogin", true);
            // and call the user page with a redirect to the doGet
            String url = response.encodeRedirectURL("./HelloLogin");
            response.sendRedirect(url);
        }
        // else, print invalid, and call doGet without setting user
        else {
            session.setAttribute("USER", null);
            session.setAttribute("firstLogin", null);
            out.println("<p><font color=\"red\">login " +
                    "invalid</font></p>");
            // and call the user login page
            doGet(request, response);
        }
    }

    public void destroy()
    {
        // do nothing, but we need this because the servlet
        // container will call it
    }
}

and the results:

one_landingpage

two_wrongentry

three_invalidlogin

four_correctentry

five_successfullogin

congratulations!

Packaging Encryption tools

•January 20, 2017 • Leave a Comment

I talked previously about security and encrypting stored passwords. I am going to expand on this and package it for use by my app, as well as add more methods to Jerry’s class.

I updated my auto-compilation script to copy (instead of move) the compiled classes where they need to go, as they need to be in the original location for compiling the referenced packages to succeed, and in the ‘classes/’ folder with the original folder structure for the web app to work. I also realized this needs to be run from within my ‘src’ folder so that the javac commands have the correct working folder.

I placed my PasswordEncryptionService.java under the ‘com/CB’ folder under the root.  Therefore, at the top of my package, I need:

package com.CB

The following work took me a lot more time than I think it should have. Mainly getting the encrypted passwords and salts in and out of a file in their raw state proved difficult. I ended up cramming the password and the salt together since both are of fixed length. I figured anyone simply looking at the file wouldn’t know that there are two things there. Even though “security through obscurity” isn’t considered to be valid security.

So, I’ll add some private class variables to store my “database”, add functions to add users, read and write the users file, print some things to the terminal for debugging purposes, and overload the authenticate method to make it a little more straight-forward and class dependent. Here is the code after I made my changes. I also made a few of Jerry’s methods use global variables I can also call (not that I expect them to change).

package com.CB;

import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import java.security.spec.InvalidKeySpecException;
import java.security.spec.KeySpec;
import java.util.*;
import java.io.*;
import java.lang.*;

import javax.crypto.SecretKeyFactory;
import javax.crypto.spec.PBEKeySpec;

public class PasswordEncryptionService {

    private static int LEN_PASS = 20;
    private static int LEN_SALT = 8;

    private String usersFile;

    private ArrayList<String> users = new ArrayList<String>();
    private ArrayList<byte[]> encryptedPasswords = new ArrayList<byte[]>();
    private ArrayList<byte[]> salts = new ArrayList<byte[]>();

    public void printBytes(byte[] bytes) {
        // print a byte array as hex pairs
        StringBuilder sb = new StringBuilder();
        sb.append(" ");
        for (byte b : bytes) {
            sb.append(String.format("%02X ", b));
        }
        System.out.print(sb.toString());
    }

    public void printArrayList(ArrayList<byte[]> a) {
        // print out a whole ArrayList. calls printBytes
        System.out.print("[");
        for(int i = 0; i < a.size(); i++) {
            printBytes(a.get(i));
            if (i < a.size()-1) {
                System.out.print(",");
            }
        }
        System.out.println("]");
    }

    public void printDatabase() {
        // print the current state of the database
        System.out.print("users: ");
        System.out.println(users);
        System.out.print("passwords: ");
        printArrayList(encryptedPasswords);
        System.out.print("salts: ");
        printArrayList(salts);

        System.out.println();
    }

    public boolean useFile(String file) {
        usersFile = file;

        return true;
    }

    public String returnFile() {
        return usersFile;
    }

    public boolean addUser(String username, String password)
            throws NoSuchAlgorithmException, InvalidKeySpecException {

        // use the provided functions to generate these values
        byte[] salt = generateSalt();
        byte[] encryptedPassword = getEncryptedPassword(password, salt);

        // if the user doesn't exist, add it, and all its values
        if (!users.contains(username)){
            users.add(username);
            encryptedPasswords.add(encryptedPassword);
            salts.add(salt);
        }

        return true;
    }

    public boolean writeFile()
            throws IOException {
        // verify there are always the same number of entries
        if ((users.size() != encryptedPasswords.size())
                || (encryptedPasswords.size() != salts.size())) {
            return false;
        }

        // write name surrounded by quotes
        // followed by encrypted password and salt which are fixed lengths
        FileOutputStream fos = new FileOutputStream(usersFile);
        for(int i=0; i<users.size(); i++) {
            fos.write('"');
            fos.write(users.get(i).getBytes());
            fos.write('"');
            fos.write(encryptedPasswords.get(i));
            fos.write(salts.get(i));
            fos.write(new String("\n").getBytes());
        }
        fos.close();

        return true;
    }

    public boolean readFile()
            throws IOException {
        //System.out.println("loading: " + usersFile + " ...");

        File file = new File(usersFile);
        if (!file.exists()) {
            // do nothing and return. No file to load
            return true;
        }

        // initialize variables for "db" read
        FileInputStream fis = new FileInputStream(file);
        int c;
        String name = "";
        byte[] pass = new byte[LEN_PASS];
        byte[] salt = new byte[LEN_SALT];
        int nl;
        boolean newline = true;

        // step through the file
        while ((c = fis.read()) != -1) {
            if (newline && (char)c == '"') {
                //should only happen for the first character of a line
                newline = false;
                continue;
            }
            if ((char)c == '"') {
                // done reading the name.
                // password and salt are a given length, so grab them next
                //System.out.println("==" + name + "==");
                fis.read(pass);
                //printBytes(pass);
                fis.read(salt);
                //printBytes(salt);
                //System.out.println();

                // add what we got from the file to the database
                users.add(name);
                encryptedPasswords.add(pass.clone());
                salts.add(salt.clone());

                // verify the next char is a newline and discard
                nl = fis.read();
                if ((char)nl != '\n') {
                    System.out.println("newline not found");
                    return false;
                }
                else {
                    name = "";
                    newline = true;
                    continue;
                }
            }

            // add the current byte to the name string
            name += String.valueOf((char)c);
        }

        fis.close();

        return true;
    }

    public boolean readFile(String filename)
            throws IOException {
        useFile(filename);
        return readFile();
    }

    public boolean authenticate(String user, String pass)
            throws NoSuchAlgorithmException, InvalidKeySpecException {
        if(!users.contains(user)) {
            return false;
        }

        int i = users.indexOf(user);
        return authenticate(pass, encryptedPasswords.get(i), salts.get(i));
    }

    public boolean authenticate(String attemptedPassword, byte[] encryptedPassword, byte[] salt)
            throws NoSuchAlgorithmException, InvalidKeySpecException {
        // Encrypt the clear-text password using the same salt that was used to
        // encrypt the original password
        byte[] encryptedAttemptedPassword = getEncryptedPassword(attemptedPassword, salt);

        // Authentication succeeds if encrypted password that the user entered
        // is equal to the stored hash
        return Arrays.equals(encryptedPassword, encryptedAttemptedPassword);
    }

    public byte[] getEncryptedPassword(String password, byte[] salt)
            throws NoSuchAlgorithmException, InvalidKeySpecException {
        // PBKDF2 with SHA-1 as the hashing algorithm. Note that the NIST
        // specifically names SHA-1 as an acceptable hashing algorithm for PBKDF2
        String algorithm = "PBKDF2WithHmacSHA1";
        // SHA-1 generates 160 bit hashes, so that's what makes sense here
        int derivedKeyLength = LEN_PASS*8; // derivedKeyLength needs to be in bits, hence *8
        // Pick an iteration count that works for you. The NIST recommends at
        // least 1,000 iterations:
        // http://csrc.nist.gov/publications/nistpubs/800-132/nist-sp800-132.pdf
        // iOS 4.x reportedly uses 10,000:
        // http://blog.crackpassword.com/2010/09/smartphone-forensics-cracking-blackberry-backup-passwords/
        int iterations = 20000;

        KeySpec spec = new PBEKeySpec(password.toCharArray(), salt, iterations, derivedKeyLength);

        SecretKeyFactory f = SecretKeyFactory.getInstance(algorithm);

        return f.generateSecret(spec).getEncoded();
    }

    public byte[] generateSalt() throws NoSuchAlgorithmException {
        // VERY important to use SecureRandom instead of just Random
        SecureRandom random = SecureRandom.getInstance("SHA1PRNG");

        // Generate a 8 byte (64 bit) salt as recommended by RSA PKCS5
        byte[] salt = new byte[LEN_SALT];
        random.nextBytes(salt);

        return salt;
    }
}

I then created a number of pure Java files to manipulate my “database” and the users file. I did not handle many of my exceptions, instead, I just passed them along (I thought it made everything cleaner). Here is the AddUsers.java file:

import java.security.NoSuchAlgorithmException;
import java.security.spec.InvalidKeySpecException;
import java.io.IOException;

import com.CB.PasswordEncryptionService;

public class AddUsers {
    public static void main(String[] args)
            throws NoSuchAlgorithmException, InvalidKeySpecException,
                IOException {
        PasswordEncryptionService pes = new PasswordEncryptionService();

        pes.useFile("/usr/share/tomcat7/webapps/Introduction/WEB-INF/" + 
                        "users.db");
        pes.readFile();

        pes.printDatabase();
        pes.addUser("Statler", "password");
        pes.addUser("Waldorf", "password");
        pes.printDatabase();

        pes.writeFile();
    }
}

and my Authenticate.java file to test that everything is working (I’m going to figure out automated testing later. For now, this is close enough.)

import java.security.NoSuchAlgorithmException;
import java.security.spec.InvalidKeySpecException;
import java.io.IOException;

import com.CB.PasswordEncryptionService;

public class Authenticate {
    public static void main(String[] args)
            throws IOException, NoSuchAlgorithmException,
                InvalidKeySpecException {
        PasswordEncryptionService pes = new PasswordEncryptionService();

        pes.readFile("/usr/share/tomcat7/webapps/Introduction/WEB-INF/" +
                "users.db");

        System.out.println(pes.authenticate("statler", "password"));
        System.out.println(pes.authenticate("waldorf", "wrong"));
    }
}

which, when run, results in:

[tomcat@ip-IP-IP-IP-IP src]$ java Authenticate
true
false

Next, I will use my new package to power my login app.

Authentication

•January 6, 2017 • Leave a Comment

There is no such thing as a 100% impenetrable security system. It should be built in layers, assuming at least one layer will get compromised. The subsequent layers will hopefully make any hacker move on to easier prey.

A good example of layered security can be shown using stored passwords. If you need authentication, and you can’t use another existing system like an existing LDAP system, you’ll have to store passwords, probably in a file. These files should have layers of security. The contents encrypted in addition to the permissions locked down. I have a system that I have root access on (it’s my system, so, obviously). In looking through files on this system (coincidentally, attempting to leverage the existing authentication system), I found my Google password (which I had entered in a web app) in plain text. The file didn’t have permissions for anyone but root, but still. I had a roommate at the time who knew more than a little bit about security. I lamented what I had found to her. She said, “It makes your teeth itch, doesn’t it?”

I want to write a basic authentication example. And I’m going to create a users/password file. Because I don’t want to use these passwords to access another system (like my unfortunate example above), I won’t even have to decrypt the passwords to plain text. It’s possible to encrypt the user-provided password and simply compare the two encrypted.

This mechanism is one I learned at UC Davis. I remember the lesson, but I needed to re-learn the implementation. So I did some searching.

I’ll create the file PasswordEncryptionService.java with the contents of Jerry on Java’s posted code. I am going to use this for the encryption methods and math. In subsequent posts, I am going to expand it to read and write the users file, and create a package that can be used by my HelloLogin app.

Merry

•December 25, 2016 • Leave a Comment

Merry Christmahanakwanzika and a happy Gregorian New Year!