Stuff I'm Up To

Technical Ramblings

JumpCloud — October 23, 2019

JumpCloud

Sometimes I’m surprised at why I’ve never come across things before. This is a big one for me. For the longest time I was pondering how to resolve some SSO requirements whilst maintaining a corporate managed directory and not spending a fortune. Traditionally this would be the infrastructure to get the likes of Azure Active Directory, ADFS, RADIUS and multi-factor authentication – and then BOOM! JumpCloud.

What I really liked about this is that I got my own directory setup in under 15 minutes and had a Linux client logging on using my SSH key. I haven’t had to do anything laborious just install the JumpCloud agent onto the machine. Once I created my user account on the cloud interface and (optionally) gave it my SSH key I was set.

The JumpCloud agent handles replicating my account to the “systems” I install the agent on. It also delivers my SSH key for me so I can connect securely to the systems I’m allocated immediately.

Auth and Management for SSO, LDAP, RADIUS, Mac, Windows, Linux, and More

As a new user I get 10 FREE accounts which is plenty to setup my own directory for home and testing. I didn’t even need a credit card.

 https://jumpcloud.com/
Distributed Virtualisation — October 21, 2019

Distributed Virtualisation

A tool box for low cost virtualisation with replication, management and high availability without the need of expensive SAN’s and shared storage devices.

http://www.ganeti.org/
https://www.synnefo.org/
https://www.linux-kvm.org/page/Main_Page
https://www.qemu.org/
https://www.linbit.com/en/drbd-community/ 
Git Credentials — September 18, 2019

Git Credentials

Using git to push commits up to the remote is all in a days work. The change happens when you switch to a new remote and use a new account.

My first actions where to change the remote for my local project. This is easy enough using git remote set-url origin [url]. It was only when I went to push this project up to the new remote repository that I found I was being denied with a 403 error, which means permission denied.

The big reason for the problem was a change from ssh to https. Using ssh was pretty straight forward, as long as you have your key and it is registered in your .gitconfig for the host your pushing to the credentials are pretty robust.

I’d take a step toward running the remotes on https due to firewall and proxy issues that meant https should be easier.

But because ssh keys can make life easier by not having a key password (cool, unless your user password is weak), the change to https means you need to provide credentials on each push.

This is where you need to start looking at Git Credentials Storage.

Under Linux you can specify a credentials file that will feed your details into the process. The file should be placed somewhere every secure and with the correct permissions to ensure it isn’t misused. For instance as a hidden file under your home directory with nly you having permission to access it.

eg.

$ touch  ~/.my-credentials
£ chmod 600 ~/.my-credentials
$ git config credential.helper 'store --file ~/.my-credentials'

But with Windows things actually get a bit easier! Which is hard for a Linux head to accept :)

The git helper for Windows means that your credentials get stored with your windows account.

$  git config credential.helper manager

Because I changed remotes and changed the account I was using, under Windows I needed to remove my old credentials. This is easy enough. I just brought up the start menu and type “credentials“. Then I chose the option for “Manage Windows Credentials“. In the list of generic credentials I could see my old account and simply removed it. The next time I pushed I was asked for new credentials which then got added into the list for me.

VSCode CRLF vs LF Battle — September 4, 2019

VSCode CRLF vs LF Battle

I’m a Linux guy. I like my line feeds a simple LF. but when developing cross platform and you hit Windows and face CRLF. It can be a real linting challenge.

Git tries to be helpful in that it translates LF to CRLF when you pull onto a windows platform. But that doesn’t help at all when your projects .eslintrc.js is set for unix type line endings.

      "linebreak-style": [
        "error",
        "unix"
      ], 

Changing CRLF to LF in VSCode is easy enough, but having to do it on every file you open is madness.

Continue reading
Guzzle and Curl — August 12, 2019

Guzzle and Curl

Related to my previous post about Laravel. Guzzle and Nginx I ran into an issue with our proxy. The proxy is always a source of fun and games.

Because the proxy breaks open SSL traffic to scan the content the clients are required to have an SSL certificate installed that tells them to trust our proxy server certificate. In Windows and Linux you can insert the CA cert into the OS using group policy or writing it into the certificate store.

Curl uses it’s own certificate store so we needed to copy the proxy CA cert into the curl store.

On Windows there wasn’t a certificate store. I created one in a location that would remain even if anything was updated or moved.

Download the cacert.pem file and place it in c:\certs. Then I just added my proxy cert in PEM on the end.

C:> type proxy.pem >> c:\certs\cacert.pem

Edit your php.ini and change the curl setting to point at the new cacert.pem file

[curl]
 curl.cainfo = c:/certs/cacert.pem

You can find what php.ini you are using with:

C:> php --ini
Configuration File (php.ini) Path: C:\windows
 Loaded Configuration File:         C:\tools\php73\php.ini
 Scan for additional .ini files in: (none)
 Additional .ini files parsed:      (none)

Restart any php service, like Apache, Nginx, Artisan, etc. and curl should then trust the proxy server.

DNSSEC Validation Failed — July 27, 2019

DNSSEC Validation Failed

Looking at my virtual dev system I noticed the time is off. I checked the timesyncd.conf and restarted timesyncd and saw lots of similar errors to this in my syslog:

Jul 25 23:18:59 buster systemd[1]: Started Network Time Synchronization.
Jul 25 23:18:59 buster systemd-resolved[357]: DNSSEC validation failed for question org IN DS: signature-expired
Jul 25 23:18:59 buster systemd-resolved[357]: DNSSEC validation failed for question org IN DNSKEY: signature-expired
Jul 25 23:18:59 buster systemd-resolved[357]: DNSSEC validation failed for question ntp.org IN DS: signature-expired
Jul 25 23:18:59 buster systemd-resolved[357]: DNSSEC validation failed for question ntp.org IN SOA: signature-expired

Initially I thought something is wrong with my DNS resolver. I then edited /etc/systemd/resolver.conf to change the DNSSEC setting by un-commenting it:

[Resolve]
#DNS=
#FallbackDNS=
#Domains=
#LLMNR=yes
#MulticastDNS=yes
DNSSEC=allow-downgrade
#DNSOverTLS=no
#Cache=yes
#DNSStubListener=yes
#ReadEtcHosts=yes

Then a restart and my time is all synced.

But then I thought about it some more. The DNSSEC was probably failing because my system time was significantly wrong by several hours. So the signature probably isn’t valid. Probably all I needed to do was set the time manually before it would sync. But a reboot sorted it and I have reset my DNSSEC back to being commented out.

Ignore Comments in Files —

Ignore Comments in Files

A very handy grep that you can use to cat your files without the hash (#) comments:

$ grep '^[^#]' /etc/systemd/timesyncd.conf

Produces only the lines that aren’t comments, eg:

[Time]
NTP=192.168.1.55 192.168.1.108
FallbackNTP=0.debian.pool.ntp.org 1.debian.pool.ntp.org 2.debian.pool.ntp.org 3.debian.pool.ntp.org

Gitlab: When an Upgrade Goes Bad! — July 25, 2019

Gitlab: When an Upgrade Goes Bad!

Today is not a lot of fun.

I’ve been seeing some issues with apt not being able to upgrade Gitlab due to a proxy error. This morning I fixed it and the upgrade from 11.11.2 to 12.1.1 began – and failed miserably!

It complained with all kinds of problems not being able to carry out migrations:

Exception: Your database is missing the 'cache_invalidation_event_id' column from the 'geo_event_log' table that is present for GitLab EE.
 Even though it looks like you're running a CE installation, it appears
 you may have installed GitLab EE at some point. To migrate to GitLab 12.0:
 Install GitLab 11.11.3 EE
 Install GitLab 12.0.x CE 

This was just the start of my problems.

Continue reading
MySQL CSV Import — July 24, 2019

MySQL CSV Import

A little while ago I wrote a php routine to import CSV files that contain a lot of data into a MySQL table. It works, but it takes it’s time doing so.

I was originally reading each line using a CSV parser and then writing each line into the table. I have 28+ files each containing 200MB of data in a million rows in each file.

Now for a change.

Continue reading
VSCode rest-client — July 15, 2019
Adding a Gnome Favourite —

Adding a Gnome Favourite

I was trying to add the Postman app to my Gnome favourite bar, but right clicking it doesn’t give me the option to add it as a favourite.

Apparently if your application is not seen as an Activity then it can’t be added. Usually I’d just create a .desktop file and use that with a launcher. But Gnome doesn’t really work like that.

I found the answer was to create my .desktop file and then copy/move it to /usr/share/applications or if it is a user specific application ~/.local/share/applications.

$ gnome-desktop-item-edit ~/Desktop/ --create-new
$ sudo mv ~/Desktop/Postman.desktop /usr/share/application 

Then I can use the “Activities” to search for the app and can now add it as a favourite.

[unixODBC][Driver Manager]Can’t open lib : file not found — July 12, 2019

[unixODBC][Driver Manager]Can’t open lib : file not found

I have no idea how we came up against this issue on one of the development images. I’d prepared it all up to the point of delivering php. After following my instructions to install the MS SQL drivers everything looked to go well, but when serving up our Laravel project in artisan PHP came up with this error message.

[unixODBC][Driver Manager]Can’t open lib ‘/opt/microsoft/msodbcsql17/lib64/libmsodbcsql-17.3.so.1.1’ : file not found

Now we have seen this before and it related to locale’s so we tried that fix and still didn’t get it to work.

Trawling the internet I came across something pointed us to use ldd to look at the .so file and check out it’s dependencies.

Continue reading