Written on 2020-06-07


Background

Using LSP (Language Server Protocol) in Emacs brings some of the convenience items present in contemporary IDE's into the Emacs environment that we are all familiar with. Clojure has proven to me to be similar from a development standpoint where it is a relatively small core language with a large number of supporting libraries, similar in many ways to Common LISP.

Having the ability to leverage Emacs with LSP for Clojure development was a really high item on my list of integrations as I saw what clojure-lsp does in a FreeBSD and Debian Linux environment and I wanted to have those capabilities available to me in OpenBSD.

The Problem

The main issue faced when working with OpenBSD is generally its compatibility with various libraries used frequently in the Linux community. Linux users generally are happy when they get their code working in that environment, stopping at that point with porting to other environments. Sometimes FreeBSD will be ported to, but OpenBSD is generally left to fend for itself. Also the OpenBSD environment is not one that is as easy to get working due to its more locked down state.

In the case of clojure-lsp, the default binary found at https://github.com/snoe/clojure-lsp does not include support for OpenBSD for the required sqlite-jdbc driver within the download provided. The sqlite DB is used by LSP for maintaining the details of a project in a quickly accessible location I believe. (Disclaimer I haven't looked at the code to confirm this, it's an assumption) Once Emacs has been configured to leverage clojure-lsp, it tries to create a sqlite DB in a directory ".lsp" at the root of the project. When this is attempted in OpenBSD, the sqlite-jdbc driver isn't contained in the shipped binary, so even though the OS is detected, it doesn't have the proper jdbc binary available to make the connection to the DB, and fails. In Emacs it simply appears at the bottom of the screen as "Connecting" and continues to sit there spinning.

Luckily lsp-clojure by default logs to a file /tmp/lsp.out that contained information regarding this failure, giving us a clue to what is happening. Below is an example of the error.

{:type java.lang.Exception
 :message No native library is found for os.name=OpenBSD and
 os.arch=x86_64. 
 path=/org/sqlite/native/OpenBSD/x86_64
 :at [org.sqlite.SQLiteJDBCLoader loadSQLiteNativeLibrary
 SQLiteJDBCLoader.java 333]}

Its pretty apparent from the error that we have a library missing in the shipped binary for

Lets Make this Work

Getting lsp-clojure functional in OpenBSD requires a couple of things to happen.

  • Rebuild sqlite-jdbc with support for OpenBSD
  • Rebuild clojure-lsp with support for the sqlite-jdbc driver compiled

The next section walks through the process

Prerequisites

  • Download the master zip file from the git repo for sqlite-jdbc: https://github.com/xerial/sqlite-jdbc
  • Clone the git repo for clojure-lsp: https://github.com/snoe/clojure-lsp
  • Ensure the following packages are installed for compiling
    • bash
    • gmake
    • lein
    • maven
  • Emacs has been properly configured for LSP and lsp-clojure as noted on their respective web-sites.

Compiling an OpenBSD version of the sqlite-jdbc driver

  • Unzip sqlite-jdbc-master.zip
  • Edit the Makefile so it only compiles native
    • modify the line as shown in Makefile
#native-all: native win32 win64 mac64 linux32 linux64 linux-arm
#linux-armv7 linux-arm64 linux-android-arm linux-ppc64 alpine-linux64

native-all: native
  • Using gmake compile sqlite-jdbc
    • The native compile will only build the host OS version
    • It was not possible to get the other OS versions to build easily, so they were ignored.
% gmake

Once its built, we need to have this installed in the local .m2 repository of the user's home directory. This is needed as when we build clojure-lsp we need to have this driver available. We use maven to perform the installation locally. This will run through some tests as part of the install process and at the end of the process an INFO line shows where it was installed.

$ mvn install 

[INFO] Scanning for projects...
[INFO] 
[INFO] -----------------------< org.xerial:sqlite-jdbc >-----------------------
[INFO] Building SQLite JDBC 3.31.2-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------

[INFO] --- maven-install-plugin:2.4:install (default-install) @ sqlite-jdbc ---
[INFO] Installing
/home/roger/Downloads/sqlite-jdbc-master/target/
sqlite-jdbc-3.31.2-SNAPSHOT.jar 
to /home/roger/.m2/repository/org/
xerial/sqlite-jdbc/3.31.2-SNAPSHOT/sqlite-jdbc-3.31.2-SNAPSHOT.jar
[INFO] Installing /home/roger/Downloads/sqlite-jdbc-master/pom.xml to 
/home/roger/.m2/repository/org/xerial/sqlite-jdbc/3.31.2-SNAPSHOT/
sqlite-jdbc-3.31.2-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  11.980 s
[INFO] Finished at: 2020-06-07T15:49:36-04:00
[INFO] ------------------------------------------------------------------------

Now that the driver is ready and installed, go to the git cloned directory for clojure-lsp and make the following changes to the deps.edn file. Make sure that the versions being built match your individual versions. By default Leiningen will look to the .m2 directory of the user running the build first, if it doesn't find the file there, it the searches these: clojars.org and Maven Central. By installing the library to the local cache, it will not go out to look for it. If it does try to look outside of the local repo, obviously this will fail. 

;;org.xerial/sqlite-jdbc {:mvn/version "3.21.0.1"}
org.xerial/sqlite-jdbc {:mvn/version "3.31.2-SNAPSHOT"}

Now its time to build a new version of clojure-lsp with support for OpenBSD. By default lein does not allow SNAPSHOTS in release build libraries as dependencies, it will trigger an error. If a SNAPSHOT was used as above, its necessary to tell Leiningen this is OK before we build clojure-lsp.

% cd <CLOJURE_LSP_DIR>
% LEIN_SNAPSHOTS_IN_RELEASE=1
% export LEIN_SNAPSHOTS_IN_RELEASE
% <PATH TO LEIN> bin
WARNING: You have $CLASSPATH set, probably by accident.
It is strongly recommended to unset this before proceeding.
WARNING: seqable? already refers to: #'clojure.core/seqable? in
namespace: 
clojure.core.incubator, being replaced by: #'clojure.core
.incubator/seqable?
WARNING: seqable? already refers to: #'clojure.core/seqable? in
namespace: 
clostache.parser, being replaced by: #'clojure.core.incub
ator/seqable?
Compiling 1 source files to /home/roger/src/clojure/clojure-lsp/target/classes
Compiling clojure-lsp.clojure-core
Compiling clojure-lsp.crawler
Compiling clojure-lsp.db
Compiling clojure-lsp.handlers
Compiling clojure-lsp.interop
Compiling clojure-lsp.main
Compiling clojure-lsp.parser
Compiling clojure-lsp.refactor.edit
Compiling clojure-lsp.refactor.transform
Compiling clojure-lsp.shared
Created /home/roger/src/clojure/clojure-lsp/target/
clojure-lsp-release-20200514T134144.jar
Created /home/roger/src/clojure/clojure-lsp/target/
clojure-lsp-release-20200514T134144-standalone.jar
Creating standalone executable: /home/roger/src/clojure/clojure-lsp/
target/clojure-lsp
Re-aligning zip offsets

Copy the clojure-lsp file created as "standalone executable" to a location available in your PATH within your .emacs.

Open Emacs and load a clojure file from a project into Emacs.

When this is performed, it should launch the lsp process and you will see some message indicating STATUS:starting at the bottom of the Emacs window. After some time.. depending upon the processor, maybe 20-30 seconds, that should change to: 

LSP :: clojure-lsp:\<PID\> initialized successfully

If a message like this appears, you did not set the environment variable properly

Release versions may not depend upon snapshots. 
Freeze snapshots to dated versions or set the
LEIN_SNAPSHOTS_IN_RELEASE environment variable to override.

Congratulations, you now have clojure-lsp working in OpenBSD!

If this doesn't happen and you have failures, or you see it sitting in the "starting" state for a long time, check the /tmp/lsp.out file to see what the error is.

When the driver is in place, once the nrepl server starts, it notices the DB was not present in the file system. Note, the only time that this particular warning was logged was when the system was able to load the OpenBSD version of the sqlite-jdbc driver.

INFO  clojure-lsp.main: :initialize 55577 ()
INFO  clojure-lsp.main: Shutting down
INFO  clojure-lsp.main: Server started
INFO  clojure-lsp.main: ====== LSP nrepl server started on port 32021
WARN  clojure-lsp.main: Initialize
WARN  clojure-lsp.db: Could not load db [SQLITE_ERROR] SQL error or
missing database (no such table: project)

Enjoy your new installation!

Until the next blog, don't talk about it...

Shut up and Hack!


    
Unless otherwise credited all material © Wilcis.com by Roger Williams