November 27, 2018

Postfix relay through Outlook (TLS)

Abstract

We take email for granted. It seems so easy. Just click the send button. But spam has made sending and receiving email a lot more complicated. This is especially true for people who like to run their own networks at home. If you are like me, you might have 1 or more physical machines running multiple virtual machines, not to mention multiple containers as well. All this network infrastructure needs to communicate with you somehow. After all, how do you keep tabs on whether or not your CRON jobs are successful? Email is the way. But sending email out from a home network without it being blocked at various points along the way can get complicated. We can go through the cost of buying a domain and email service, but who wants the cost? So how do we get the email through? This post explains how to configure Postfix to relay email through an Outlook account over TLS.

Disclaimer

This post is solely informative. Critically think before using any information presented. Learn from it but ultimately make your own decisions at your own risk.

Requirements

I did all of the work for this post using the following major technologies. You may be able to do the same thing with different technologies or versions, but no guarantees.

  • An Outlook account (outlook.com, live.com, etc.)
  • Ubuntu 18.04.1 LTS
  • Postfix 3.3.0

Install Postfix

I’m not going to go into a lot of explanation. I’m simply going to state the steps and configuration I did which got everything working.

First thing you need to do is install Postfix.

$ sudo apt-get update
$ sudo apt-get install postfix mailutils

NOTE For the Postfix installation, when it asks for the “System mail name” value, it should be the same as the name of the server.

Configure Postfix Authentication for Outlook

Configure Postfix to login to Outlook’s SMTP server. This configures the server name, port, account id, and clear text password.

# Create the password file
$ cd /etc/postfix/sasl
$ touch sasl_passwd_outlook
$ chmod 600 sasl_passwd_outlook

Now edit the sasl_passwd_outlook file using your favorite editor. Make it look like Listing 1, replacing ACCOUNT_NAME and CLEAR_TEXT_PASSWD appropriately.

Listing 1 - sasl_passwd_outlook

[smtp-mail.outlook.com]:587 ACCOUNT_NAME@outlook.com:CLEAR_TEXT_PASSWD

Now hash the sasl_passwd_outlook file into a Postfix .db file

$ postmap sasl_passwd_outlook

Configure Postfix Email Mapping for Outlook

If user mike attempts to send an email from a server with the name bluegreensky, the default Postfix FROM address will be mike@bluegreensky. This is not good because if you try to relay this email through Outlook, it will be blocked since the FROM address does not match the Outlook account. To get around this, configure a regular expression mapping file that will change all local email addresses (like mike@bluegreensky) to the Outlook account email address.

First, create a generic map file that’s empty

$ mkdir /etc/postfix/map
$ cd /etc/postfix/map
$ touch generic_map
$ chmod 600 generic_map
# Hash the file into a *.db file
$ postmap /etc/postfix/map/generic_map

Next, create an Outlook map file that will change local email address to the Outlook account email address.

$ mkdir /etc/postfix/map
$ cd /etc/postfix/map
$ touch regex_map_outlook
$ chmod 600 regex_map_outlook

Now edit the regex_map_outlook file using your favorite editor. Make it look like Listing 2, replacing HOSTNAME - just HOSTNAME, not @HOSTNAME…don’t lose the @ character - and ACCOUNT_NAME appropriately.

Listing 2 - regex_map_outlook

/.+@HOSTNAME/    ACCOUNT_NAME@outlook.com

Now hash the regex_map_outlook file into a Postfix .db file

$ postmap regex_map_outlook

Configure Postfix TLS for Outlook

TLS must be used to connect to Outlook SMTP servers. During Postfix installation, a main.cf is created. It must be edited

$ cd /etc/postfix

Now edit the main.cf file using your favorite editor. In Listing 3 are the values you must add or update in main.cf. For each of the name/value pairs below, search main.cf to see if it already exists. If so, use my value below. If not, add my value to the end of main.cf.

Listing 3 - main.cf

# Outlook
relayhost = [smtp-mail.outlook.com]:587
smtp_use_tls = yes
smtp_sasl_auth_enable = yes
smtp_sasl_security_options =
smtp_sasl_password_maps = hash:/etc/postfix/sasl/sasl_passwd_outlook
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
smtp_tls_protocols = !SSLv2,!SSLv3,!TLSv1.1
smtp_tls_ciphers = medium
smtp_tls_mandatory_ciphers = medium
smtp_tls_security_level = may
smtp_generic_maps = hash:/etc/postfix/map/generic_map, regexp:/etc/postfix/map/regex_map_outlook

Restart Postfix

Restart Postfix so it picks up all the new configuration.

bash $ service postfix restart

Testing Postfix

If all goes well, Postfix restarted without any errors and is now configured to relay email from the server through the Outlook account. I typically test this with both mail and at.

# Test sending mail directly
$ echo "test email message" | mail -s "test email from server" some_email_address@someprovider.com
# Test sending mail through scheduler at
$ echo "echo \"gosh golly, it is AT\"" | at now

Do this testing, and see if you get the email your are expecting.

NOTE When testing with at, you will want to create a ~/.forward file with your email address so the results of the scheduled job are not delivered to your server’s local account’s inbox.

Summary

This post shows how to configure Postfix to relay email through an Outlook account over TLS. Sending email from a home network and have it not be blocked is tough. This configuration got me working. I hope it works for you.

November 26, 2018

Postfix relay through Yahoo! (SSL)

Abstract

We take email for granted. It seems so easy. Just click the send button. But spam has made sending and receiving email a lot more complicated. This is especially true for people who like to run their own networks at home. If you are like me, you might have 1 or more physical machines running multiple virtual machines, not to mention multiple containers as well. All this network infrastructure needs to communicate with you somehow. After all, how do you keep tabs on whether or not your CRON jobs are successful? Email is the way. But sending email out from a home network without it being blocked at various points along the way can get complicated. We can go through the cost of buying a domain and email service, but who wants the cost? So how do we get the email through? This post explains how to configure Postfix to relay email through a Yahoo! account over SSL.

Disclaimer

This post is solely informative. Critically think before using any information presented. Learn from it but ultimately make your own decisions at your own risk.

Requirements

I did all of the work for this post using the following major technologies. You may be able to do the same thing with different technologies or versions, but no guarantees.

  • A Yahoo! account
  • Ubuntu 18.04.1 LTS
  • Postfix 3.3.0

Install Postfix

I’m not going to go into a lot of explanation. I’m simply going to state the steps and configuration I did which got everything working.

First thing you need to do is install Postfix.

$ sudo apt-get update
$ sudo apt-get install postfix mailutils

NOTE For the Postfix installation, when it asks for the “System mail name” value, it should be the same as the name of the server.

Configure Yahoo! Account

The Yahoo! Account’s security setting must be set to allow password authentication. By default, it is not enabled. Go to the Yahoo! Account Security page and make sure “Password is enabled” is set as showing in Figure 1.

Figure 1 - Yahoo! Account Security

Yahoo! Account Security
Yahoo! Account Security

Configure Postfix Authentication for Yahoo!

Configure Postfix to login to Yahoo!’s SMTP server. This configures the server name, port, account id, and clear text password.

# Create the password file
$ cd /etc/postfix/sasl
$ touch sasl_passwd_yahoo
$ chmod 600 sasl_passwd_yahoo

Now edit the sasl_passwd_yahoo file using your favorite editor. Make it look like Listing 1, replacing ACCOUNT_NAME and CLEAR_TEXT_PASSWD appropriately.

Listing 1 - sasl_passwd_yahoo

[smtp.mail.yahoo.com]:465 ACCOUNT_NAME@yahoo.com:CLEAR_TEXT_PASSWD

Now hash the sasl_passwd_yahoo file into a Postfix .db file

$ postmap sasl_passwd_yahoo

Configure Postfix Email Mapping for Yahoo!

If user mike attempts to send an email from a server with the name bluegreensky, the default Postfix FROM address will be mike@bluegreensky. This is not good because if you try to relay this email through Yahoo!, it will be blocked since the FROM address does not match the Yahoo! account. To get around this, configure a regular expression mapping file that will change all local email addresses (like mike@bluegreensky) to the Yahoo! account email address.

First, create a generic map file that’s empty

$ mkdir /etc/postfix/map
$ cd /etc/postfix/map
$ touch generic_map
$ chmod 600 generic_map
# Hash the file into a *.db file
$ postmap /etc/postfix/map/generic_map

Next, create a yahoo map file that will change local email address to the Yahoo! account email address.

$ mkdir /etc/postfix/map
$ cd /etc/postfix/map
$ touch regex_map_yahoo 
$ chmod 600 regex_map_yahoo 

Now edit the regex_map_yahoo file using your favorite editor. Make it look like Listing 2, replacing HOSTNAME - just HOSTNAME, not @HOSTNAME…don’t lose the @ character - and ACCOUNT_NAME appropriately.

Listing 2 - regex_map_yahoo

/.+@HOSTNAME/    ACCOUNT_NAME@yahoo.com

Now hash the regex_map_yahoo file into a Postfix .db file

$ postmap regex_map_yahoo

Configure Postfix SSL for Yahoo!

SSL must be used to connect to Yahoo! SMTP servers. During Postfix installation, a main.cf is created. It must be edited

$ cd /etc/postfix

Now edit the main.cf file using your favorite editor. In Listing 3 are the values you must add or update in main.cf. For each of the name/value pairs below, search main.cf to see if it already exists. If so, use my value below. If not, add my value to the end of main.cf.

Listing 3 - main.cf

# Yahoo!
relayhost = [smtp.mail.yahoo.com]:465
smtp_use_tls = yes
smtp_sasl_auth_enable = yes
smtp_sasl_security_options =
smtp_sasl_password_maps = hash:/etc/postfix/sasl/sasl_passwd_yahoo
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
smtp_tls_wrappermode = yes
smtp_tls_security_level = encrypt
smtp_generic_maps = hash:/etc/postfix/map/generic_map, regexp:/etc/postfix/map/regex_map_yahoo

Restart Postfix

Restart Postfix so it picks up all the new configuration.

bash $ service postfix restart

Testing Postfix

If all goes well, Postfix restarted without any errors and is now configured to relay email from the server through the Yahoo! account. I typically test this with both mail and at.

# Test sending mail directly
$ echo "test email message" | mail -s "test email from server" some_email_address@someprovider.com
# Test sending mail through scheduler at
$ echo "echo \"gosh golly, it is AT\"" | at now

Do this testing, and see if you get the email your are expecting.

NOTE When testing with at, you will want to create a ~/.forward file with your email address so the results of the scheduled job are not delivered to your server’s local account’s inbox.

Summary

This post shows how to configure Postfix to relay email through a Yahoo! account over SSL. Sending email from a home network and have it not be blocked is tough. This configuration got me working. I hope it works for you.

November 09, 2018

Derby Database Backup

Abstract

I have already posted a number of blogs about Derby:

This wasn’t intended to be a series. But over the years I’ve been using Derby more and more. Recently, I started using Derby as my database of choice for my Microservice architecture. These are personal-use applications, so Derby is more than sufficient. Even though these are personal-use applications, I require multiple servers with limited user permissions and - most importantly - backup. I’d hate to lose my data! The purpose of this post is to demonstrate how to backup a Derby database.

Disclaimer

This post is solely informative. Critically think before using any information presented. Learn from it but ultimately make your own decisions at your own risk.

Requirements

I did all of the work for this post using the following major technologies. You may be able to do the same thing with different technologies or versions, but no guarantees.

  • Apache Derby 10.14.2.0
  • OpenJDK 64-Bit Server VM Zulu11.1+23 (build 11-ea+22, mixed mode)

Download

There are no downloads with this blog post. The scripts are shown in full.

Derby System Utility

Backing up a Derby database is really quite simple. Derby has a built-in system utility for performing the backup. The utility is SYSCS_UTIL.SYSCS_BACKUP_DATABASE('/location/of/the/backup/'). When called, Derby will lock the database and perform the copy operation to the file system location you specify as the parameter to SYSCS_BACKUP_DATABASE. Now that we know the system utility to do the backup, let’s look at a bash script to automate it.

Backup Script

Listing 1 is a bash script which can be easily modified to backup any Derby database on any network server.

Listing 1 - derby-mydatabase-backup.sh

#!/bin/bash

# Define a bunch of variables which will be used within this script.
# The names of the variables should be self-explanatory.
DERBY_HOME=/opt/db-derby-10.14.2.0-bin/
NETWORK_SERVER_HOST=localhost
NETWORK_SERVER_PORT=1527
DATABASE_NAME=mydatabase
DATABASE_USER=sa
DATABASE_PASSWORD=abc123
JDBC_URL="jdbc:derby://$NETWORK_SERVER_HOST:$NETWORK_SERVER_PORT/$DATABASE_NAME"
BACKUP_DIRECTORY="/tmp/$DATABASE_NAME-backup/$NETWORK_SERVER_PORT"
BACKUP_SCRIPT="$BACKUP_DIRECTORY/backup.sql"

# Remove old backup if it exists. It is not a good idea to
# perform a backup on top of an existing backup.
rm -rf $BACKUP_DIRECTORY
mkdir -p $BACKUP_DIRECTORY
cd $BACKUP_DIRECTORY

# Use the `echo` command to dynamically create an SQL file.
# This SQL file will be used by Derby `ij` to connect to
# the database and perform the backup.
echo "connect '$JDBC_URL' user '$DATABASE_USER' password '$DATABASE_PASSWORD';" >> $BACKUP_SCRIPT
echo "CALL SYSCS_UTIL.SYSCS_BACKUP_DATABASE('$BACKUP_DIRECTORY');" >> $BACKUP_SCRIPT
echo "exit;" >> $BACKUP_SCRIPT

# Run the Derby `ij` application, passing it the SQL file
# that was just dynamically created. `ij` will read the 
# SQL file, executing its commands. This will then
# cause `ij` to connect to the database and call the 
# system utility to perform the backup.
$DERBY_HOME/bin/ij $BACKUP_SCRIPT

Let’s take a look at this script in more detail.

Lines 5–15 setup a number of variables used within the script. Some variables are used to set the values of other variables. There is nothing too complicated here. The names of the variables are self-explanatory.

Lines 17–19 is file system maintenance. It is not a good idea to perform a backup on top of an existing backup. So these lines remove an existing backup (if it exists) and creates a new, empty, backup directory.

Lines 24–26 are then responsible for creating the backup.sql script file. This script file contains the SQL commands to perform the backup. Line 24 is the connect command so Derby ij can connect to the database you want to backup. Line 25 is where the magic happens with a call to the SYSCS_BACKUP_DATABASE system utility. The location of the backup directory is passed as a parameter to the utility. When this SQL command is executed, Derby will lock the database and perform the backup. Line 26 is the exit command to exit ij.

Line 33 is then finally where everything happens. The Derby ij command is called with the location of the dynamically created backup.sql file passed to ij as a command-line parameter. When bash executes line 33, and if everything goes well, the Derby database will be backed up.

NOTE If you are running the Derby network server with a Java security policy, you may run into some problems with this script. the Java SecurityManager may prevent the network connection to the database or the SecurityManager my encounter permission problems writing to the backup directory.

Summary

Backing up a Derby database is pretty easy. Just call SYSCS_UTIL.SYSCS_BACKUP_DATABASE('/location/of/the/backup/').

References

Backing Up a Database. (2013, January 24). db.apache.org. Retrieved from https://db.apache.org/derby/docs/10.0/manuals/admin/hubprnt43.html.

August 23, 2018

Multiple Derby Network Servers on the same Host

Abstract

Suppose you want to start multiple Derby network servers on the same host. They need to be listening on different ports and ideally store their data in different locations. The listings below show Linux bash and Windows batch scripts to configure starting a Derby network server. In this example, the Derby network server will listen on port 1110. Each Derby network server will also have its own file system location to store its databases.

Disclaimer

This post is solely informative. Critically think before using any information presented. Learn from it but ultimately make your own decisions at your own risk.

Requirements

I did all of the work for this post using the following major technologies. You may be able to do the same thing with different technologies or versions, but no guarantees.

  • Apache Derby 10.14.1.0
  • Java zulu11.1+23-ea-jdk11

I am not going to go through the process of downloading and installing these technologies. I’ll leave that as an exercise for you.

Linux bash scripts

Here are the Linux bash scripts to configure starting a Derby network server. We’ll look at the details of each file. First we’ll set the environment.

Listing 1 - setenv.sh

#!/bin/bash
 
export DERBY_HOME=/home/derby/opt/derby
export PATH="$DERBY_HOME/bin:$PATH"
echo "DERBY_HOME=$DERBY_HOME"

export JAVA_HOME=/home/derby/opt/java
echo "JAVA_HOME=$JAVA_HOME"

export NS_HOME=/var/local/derby/1110
mkdir -p $NS_HOME
echo "NS_HOME=$NS_HOME"

export NS_PORT=1110
echo "NS_PORT=$NS_PORT"

export NS_HOST=0.0.0.0
echo "NS_HOST=$NS_HOST"
 
export DERBY_OPTS=""
export DERBY_OPTS="$DERBY_OPTS -Dderby.drda.host=$NS_HOST"
export DERBY_OPTS="$DERBY_OPTS -Dderby.drda.portNumber=$NS_PORT"
export DERBY_OPTS="$DERBY_OPTS -Dderby.system.home=$NS_HOME"

DERBY_HOME is self explanatory: it’s where Derby is unzipped (installed). Add Derby’s bin directory to the PATH.

JAVA_HOME is self explanatory: it’s where Java is unzipped (installed). Add Java’s bin directory to the PATH.

NS_HOME is “Network Server Home”. This is the directory the Derby network server will use to store its configuration and databases. Whenever a new database is created on this Derby network server, a new sub-directory will be created under NS_HOME for the new database. This allows multiple Derby network servers running on the same host to keep their data separate.

NS_PORT is self explanatory: it’s the port the Derby network server uses to listen for connections. This allows multiple Derby network servers to run on the same host.

NS_HOST sets the network interface used by the Derby network server when listening for connections. By default, the Derby network server only listens for connections on the loopback address of 127.0.0.1. This default means clients must run on the same host as the network server - not very useful. By setting the host to 0.0.0.0, the Derby network server will listen for connections on any network interface on the host. If your Derby network server host has multiple network interfaces, NS_HOST should be set to the IP of one of those interfaces. Setting this value allows clients to be remote and run on any host.

DERBY_OPTS is the system property used to get all of the configuration options to Derby. It’s value is created by concatenating together the appropriate Derby system properties with their associated values.

Now that the environment is set, we can start the server.

Listing 2 - start.sh

#!/bin/bash

# Directory of the script
SD=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )

# Source in common variables
source $SD/setenv.sh

startNetworkServer -noSecurityManager

SD is Script Directory. The evaluation determines the fully-qualified file system location of the start.sh script and assigns it to SD. This is useful when referencing other scripts.

source self explanatory: it sources in the environment configuration to run the Derby network server.

startNetworkServer -noSecurityManager starts the Derby network server. The DERBY_OPTS variable - set in setenv.sh - is used to configure the network server. The -noSecurityManager command line option needs some explanation. By default, Derby’s Java process runs with a limited security policy. I’ve found that this limited security policy gets in the way of database operations which I normally would just expect to work. By specifying -noSecurityManager, you run the Derby network server without any security policy. This may not be the ideal way for you to run a Derby network server. However, this blog is limited in scope to running the server. Search for my other blog on how to secure a Derby network server.

Now that the server is running, we need to stop it.

Listing 3 - stop.sh

#!/bin/bash

# Directory of the script
SD=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )

# Source in common variables
source $SD/setenv.sh

stopNetworkServer

All of this is self explanatory. No further comments are needed for this script.

Windows batch scripts

Here are the Windows batch scripts to configure starting a Derby network server. We will not look at these files in additional details; details are included with the Linux bash scripts.

Listing 1 - setenv.cmd

@echo off
 
set DERBY_HOME=C:\Applications\Derby
set PATH=%DERBY_HOME%\bin;%PATH%
echo DERBY_HOME=%DERBY_HOME%

set JAVA_HOME=C:\Applications\Java
echo JAVA_HOME=%JAVA_HOME%

set NS_HOME=C:\Users\Derby\1110
md %NS_HOME% 2>NUL
echo NS_HOME=%NS_HOME%

set NS_PORT=1110
echo NS_PORT=%NS_PORT%

set NS_HOST=0.0.0.0
echo NS_HOST=%NS_HOST%
 
set DERBY_OPTS=-Dderby.drda.host=%NS_HOST%
set DERBY_OPTS=%DERBY_OPTS% -Dderby.drda.portNumber=%NS_PORT%
set DERBY_OPTS=%DERBY_OPTS% -Dderby.system.home=%NS_HOME%

Listing 2 - start.cmd

@echo off
call setenv.cmd
startNetworkServer -noSecurityManager

Listing 3 - stop.cmd

@echo off
call setenv.cmd
stopNetworkServer

Summary

That’s it. I hope you enjoyed learning how to run multiple Derby network servers on the same host.

May 23, 2018

Apache Derby Database Users and Permissions

Abstract

Apache Derby is awesome! Especially in a Microservices environment where the data for services (may) shrink and not require a heartier RDBMS. Derby is awesome because it’s so easy to use, especially when it come to users and permissions - you don’t need any! But, it may be the case you want to create an application-level user with limited permissions to use in Derby. The purpose of this blog is to document how to create application-level, limited permission users in Derby.

Disclaimer

This post is solely informative. Critically think before using any information presented. Learn from it but ultimately make your own decisions at your own risk.

Requirements

I did all of the work for this post using the following major technologies. You may be able to do the same thing with different technologies or versions, but no guarantees.

  • Apache Derby 10.14.1.0
  • Java 1.8.0_152_x64

I am not going to go through the process of downloading and installing these technologies. I’ll leave that as an exercise for you.

Run Derby Network Server

The first thing you must do is run a Derby network server. In my previous blog post titled Multiple Derby Network Servers on the same Host, I give detailed instructions on how to do this. Please refer to that blog post to run your own Derby network server.

Configure Derby Network Server

To configure the Derby network server, you need to create a derby.properties file. But where does the file go? It can go in a couple different places. Let’s take a look.

I’ll first assume that you ignored the Run Derby Network Server section above and instead are running Derby with all its defaults. If that’s the case, you probably started the network server by finding the %DERBY_HOME%\bin\startNetworkServer.bat file and double-clicking it. If you did this - highly not recommended - then Derby thinks the %DERBY_HOME%\bin directory is its system directory. You can confirm this by looking for the %DERBY_HOME%\bin\derby.log file. If confirmed, then you need to create a %DERBY_HOME%\bin\derby.properties file. Wherever the derby.log file is, that’s where you create the derby.properties file.

On the other hand if you didn’t ignore the Run Derby Network Server section above, congratulations! The derby.properties file must go into the directory set by the -Dderby.system.home Java system property.

Now that you know where to put the derby.properties file, here is (an example) of what to put in it:

# Passwords don't expire for 10 years
derby.authentication.native.passwordLifetimeMillis=315360000000

# Use the best hash algorithm you can
derby.authentication.builtin.algorithm=SHA-512

# Use a larger salt length for better security
derby.authentication.builtin.saltLength=128

# Re-hash this number of times for better security
derby.authentication.builtin.iterations=1564

The first property derby.authentication.native.passwordLifetimeMillis is the most important one. It configures how long Derby user passwords can be used before they go stale. By default, a password goes stale in 31 days unless this property is configured.

NOTE Derby uses the word “stale”, whereas we would typically use the word “expired”.

The value of this property is in milliseconds. The example above - 315360000000 - means a user’s password goes stale 10 years after the user is created. This configuration is for all users in every database in that Derby network server. There does not seem to be any per-user password expire setting.

Now you have the derby.properties file in place. Start the network server and let’s use it.

Run ij

ij is to Derby what sqlplus is to Oracle; just a simple command-line interface to execute SQL. Find and run %DERBY_HOME%\bin\ij.bat. For the rest of the blog, the "ij>" prompt will indicate SQL commands that must be executed within ij.

With ij running, we can create a database. Let’s do that next.

Create a Database

First, make sure your Derby network server is configured (as described above), and running (as described above) and is waiting for connections on whatever port you configured.

Second, make sure ij is running. You should be at this prompt:

ij>

Third, issue a connect statement which will create a new database.

ij> connect 'jdbc:derby://localhost:11528/resiste;create=true;' user 'sa_resiste';

Let’s look at this in more detail. localhost assumes the Derby network server is running on the same machine as ij, though it can be any host on your network. 11528 is the port the Derby network server is using to listen for connections. resiste is the name of the database to connect to; remember a Derby network server can have many different databases. create=true tells Derby to create the database if it doesn’t exist already. When created, the resiste database will the file system folder %derby.system.home%\resiste. sa_resiste is name of the admin user for the resiste database. Usually it’s just sa. Next we’ll look at setting the password for this admin user.

Create the Admin User

Derby provides an internal system call to create users and passwords. Make sure you are connected to the resiste database then execute:

ij> CALL SYSCS_UTIL.SYSCS_CREATE_USER('sa_resiste', 'derby123'); 
ij> disconnect;
ij> exit;

Line 1 creates the sa_resiste user with the password derby123. Lines 3 and 4 then disconnect from the database and exits ij.

RESTART THE NETWORK SERVER NOW

After restarting, let’s see if it worked. Connect with sa_resiste and no password. Connection will get authentication failure.

ij> connect 'jdbc:derby://localhost:11528/resiste' user 'sa_resiste';
ERROR 08004: Connection authentication failure occurred.  Reason: Userid or password invalid.

Now connect with sa_resiste and password. Connection will succeed.

ij> connect 'jdbc:derby://localhost:11528/resiste' user 'sa_resiste' password 'derby123';
ij> 

Good! The admin user is now created. Next we’ll use the admin user to create a table. This table will be used to validate the permissions of the application-level user we’ll create later.

Create Test Table

Now we are going to use the admin user to create a test table. We will do this for a couple reasons.

  1. Verify the admin user has all permissions and is able to execute these SQL commands.
  2. Verify the permissions of the application-level user we’ll create later.
ij> connect 'jdbc:derby://localhost:11528/resiste' user 'sa_resiste' password 'derby123';
ij> create schema testing;
ij> set schema testing;
ij> create table names (full_name varchar(100));
ij> insert into names values ('rita red');
ij> select * from names;
FULL_NAME
----------------------------------------------------------------------------------------------------
rita red
ij> disconnect; 

Next let’s create the application-level user.

Create the Application User

Now for the fun stuff. Let’s create an application-level user. This will be a user with permission limited to only the operations an application is able to perform. For example, if your Microservice is only going to GET data, then the application-level user should only have SELECT permissions on the database table. We will test the application-level user’s permission, but first let’s create the user.

ij> connect 'jdbc:derby://localhost:11528/resiste' user 'sa_resiste' password 'derby123';
ij> CALL SYSCS_UTIL.SYSCS_CREATE_USER('oscar', 'orange');
ij> disconnect;
ij> exit;

NOTE Remember, oscar’s password will go stale in 31 days by default. However, if you have a derby.properties file with the derby.authentication.native.passwordLifetimeMillis property set, then it will go stale after whatever time is set by that property. There is no other way to change the time it takes for the password to go stale.

RESTART THE NETWORK SERVER NOW

After restarting, let’s see if it worked. Connect with oscar. Connection will succeed, but, oscar won’t have the permission to read the test table.

ij> connect 'jdbc:derby://localhost:11528/resiste' user 'oscar' password 'orange';
ij> select * from testing.names;
ERROR 42502: User 'OSCAR' does not have SELECT permission on column 'FULL_NAME' of table 'TESTING'.'NAMES'.
ij> disconnect;

Even though the SELECT statement failed, failure means a successful test. oscar has no permissions so should not be able to select from the test table. Let’s configure oscar next.

Configure the Application User

Let’s set some permissions for oscar. Of course the sa_resiste admin user is required to do this.

ij> connect 'jdbc:derby://localhost:11528/resiste' user 'sa_resiste' password 'derby123';
ij> set schema testing;
ij> grant select on names to oscar;
ij> disconnect;

This will give oscar only 1 permission: to select from TESTING.NAMES table. Let’s see if it worked.

ij> connect 'jdbc:derby://localhost:11528/resiste' user 'oscar' password 'orange';
ij> select * from testing.names;
FULL_NAME
----------------------------------------------------------------------------------------------------
rita red
ij> disconnect;

Congratulations! You now have an application-level user with limited permissions in your Derby database.

Summary

I hope you enjoyed learning how to do simple user administration with Derby.

March 02, 2018

My Thoughts on Jakarta EE

The Announcement

On February 26, 2018, I saw a post on Twitter saying EE4J tallied the results of the naming survey and Jakarta EE is the new brand for the open source enterprise software standard (Milinkovich, 2018). From what I’ve seen on Twitter and read on other blogs, the Java enterprise community has been very supportive of the re-branding. I am supportive as well (I voted for Jakarta EE) and will continue to evangelize Jakarta EE. Though this is a monumental step, the challenges for EE4J and the Jakarta EE brand are far from over. I’d like to share what I think are some of those challenges. You can skip to tl;dr to save time. They are:

  1. Getting the Re-branding to Stick
  2. Release Cadence
  3. Continued Emphasis on the EE Server

Getting the Re-branding to Stick

The announcement hadn’t even been a day old when Twitter posts started from recruiters looking for Jakarta EE experience (Ament, 2018). Java Enterprise software has been around for nearly 20 years, and over that time it has gone through previous cycles of renaming and re-branding. Here they are:

J2EE (1999) -> Java EE (2006) -> Jakarta EE (2018)

Re-branding is hard, and it can be argued that neither Sun nor Oracle did a good job promoting the brands. You can search Twitter for posts joking about how it has been over 10 years since the re-brand to Java EE but recruiters still advertise and look for “J2EE”. I am guilty of this as well. My LinkedIn profile has “J2EE” all over it so I can be found.

So how will EE4J fair promoting the Jakarta EE brand? I think success can be measured by management knowing about it. It’s sometimes dangerous when managers know things. But, if managers start asking, “are we are using Jakarta EE in new projects”, that’s success in branding.

NOTE It’s OK if managers don’t know what Jarkarta EE is, as long as they know it’s the “thing to do” :)

Release Cadence

Java SE is experiencing this challenge right now. The Java community griped about how long it took for Java to evolve and release new versions. Now that Oracle announced Java SE is on a 6 month release cycle (Evens, 2017) with limited long term support options (Azul, 2018), organizations don’t seem to know what to do (be careful what you wish for). Software is moving faster than infrastructure can keep up.

Release cadence can be a problem with Jakarta EE as well. The Java enterprise community may get its wish with rapid releases of Jakarta EE. But if this happens, I don’t think the releases will get the fan fair the community expects. Infrastructure won’t be able to keep up. Organizations will keep to their multi-year upgrade plans meaning long periods of time between server upgrades. Jakarta EE developers will be stuck using old standards like Java EE developers are stuck now.

A rapid release cadence for Jakarta EE may even cause more harm than good. As it is now, if an organization is upgrading servers every few years, the perceived risk of the upgrade is somewhat reduced because the standard may change by only 1 number: Java EE 6 to 7. On paper, that doesn’t look too risky. But if there have been multiple release over the years and the change is from Jakarta EE 9 to 14, the perceived risk is now YIKES!

I think Jakarta EE can support a rapid release cadence. However, to do this it has to face what I consider to be its most daunting challenge: the server.

Continued Emphasis on the EE Server

The final and most glaring challenge EE4J faces is continuing to remain server-technology focused. Will Jakarta EE continue to focus on the standards to create an “EE Server”? I hope not!

20 years ago, the expectation was to install a “heavy” server (WebLogic, WebSphere) and be able to ride that installation with very few changes for 3–5 years and you’re all good. Even security patches over those years were very limited; it was a different era! That’s the way infrastructure worked then, and if we are honest with ourselves, it’s the way infrastructure works now. Infrastructure is a huge impediment to software development. Developers want to use the latest-and-greatest, but infrastructure does not want to “take the risk”. Upgrading server software is always painful: lots of testing, lots of things breaking. So infrastructure avoids upgrades as much as it can.

The development community has gotten around this infrastructure problem by moving functionality out of the server and into the application, hence the Spring framework. Spring is successful precisely because of this infrastructure problem. Infrastructure typically does not care what’s inside the applications running on the server - the applications can change and upgrade as much as they want - just so long as the server itself doesn’t have to change. Is your production WebSphere installation 10 years old? “No problem,” developer’s say, “We aren’t really using anything the server provides anyway.”

Jakarta EE needs to evolve into a standard for an enterprise framework vs. an enterprise server. Spring, Hibernate, and other open source project can continue to push innovation forward. Organizations that want to take the risk and use those bleeding-edge technologies can do so. Once proven, their innovations can be incorporated into the Jakarta EE standard framework for the rest of us. Jakarta EE developers can use the new features added to the framework as quickly as their next product release by a simple POM <version> update…no need to install a new server. As an added bonus, if Jakarta EE sticks to TCK requirements and backward compatibility, Jakarta EE developers will have the confidence things won’t break when they change that often feared <version> number.

If Jakarta EE evolves into a standard for an enterprise framework (I hope so), Jakarta EE applications still need to run in something. Perhaps keeping to the current trend of using the Servlet container (Tomcat, Jetty) would be sufficient. But whatever form it takes, the expectation should be that the “server” would be able to sit for 10 years with minimal updates and not get in the way of the applications running in them. When the EE4J charter was under review, I believe there was some discussion on using the word “runtime” instead of “server”. This is appropriate. Have a Jakarta EE runtime (JEER) that can run for years without requiring major upgrades.

tl;dr

I fully support Jakarta EE as a brand and will continue to evangelize the technology as an open standard for enterprise development.

Successfully branding Jakarta EE will be a challenge. Branding will be successful if managers start asking, “Are we using Jakarta EE”? Jakarta EE should be a buzz word and the “thing to use” even though people may have no idea what it is. Sound familiar?

Release cadence may backfire on Jakarta EE because upgrading server installations will continue to move a lot slower.

Jakarta EE needs to evolve into a enterprise standard framework, deployable as part of an application vs. an enterprise standard server that has to be installed or upgraded on a box. The former results in speed, agility, and innovation. The latter results in stagnation.

Thanks for reading.

References

Milinkovich, M., (2018, February 26). And the Name Is…[Blog Post]. Retrieved from https://mmilinkov.wordpress.com/2018/02/26/and-the-name-is/.

Ament, J. D., (2018, February 26). This just in [Tweet]. Retrieved from https://twitter.com/JohnAment/status/968324168264704000.

Evans, B., (2017, September 6). Java to Move to 6-Monthly Release Cadence [Blog Post]. Retrieved from https://www.infoq.com/news/2017/09/Java6Month.

Azul Systems, (2018, January 17). Azul Systems Announces Enhanced Java SE Support Plans [Blog Post]. Retrieved from https://globenewswire.com/news-release/2018/01/17/1295592/0/en/Azul-Systems-Announces-Enhanced-Java-SE-Support-Plans.html.

January 10, 2018

Bootstrap CDI 2.0 in Java SE

Abstract

This is a quick reference for bootstrapping CDI 2.0 in a Java SE application.

CDI 2.0 (JSR 365)

Maven Dependency

<dependency>
    <groupId>org.jboss.weld.se</groupId>
    <artifactId>weld-se-core</artifactId>
    <version>3.0.2.Final</version>
</dependency>

beans.xml

File location is /META-INF/beans.xml. This file is optional, though I’ve found it’s still good to have.

<beans version="2.0" 
    xmlns="http://xmlns.jcp.org/xml/ns/javaee"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
     http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/beans_2_0.xsd" 
    bean-discovery-mode="all"
>
    <scan>
        <exclude name="org.jboss.weld.**" />
    </scan>  
</beans>

Java SE Application

package org.ferris.cdi.example.main;

import javax.enterprise.inject.se.SeContainer;
import javax.enterprise.inject.se.SeContainerInitializer;

/**
 * The main() method for this application
 *
 * @author <a href="mailto:mjremijan@yahoo.com">Mike Remijan</a>
 */
public class Main {
    public static void main(String[] args) {

        SeContainer container
            = SeContainerInitializer.newInstance().initialize();

        Main main
            = container.select(Main.class).get();

        //. . .
    }
}

Summary

That’s it…enjoy!