Stuff I'm Up To

Technical Ramblings

Laravel Caching Indefinitely — June 28, 2019

Laravel Caching Indefinitely

I recently attended the Laravel UK conference and learned a lot of very useful things. One of them related to a caching methodology being used for data that rarely changes.

Up until now I’d been taking best guesses to how long to cache data for on a case by case basis. In one particular example I reckoned 8 hours was about right as the data only really changed once every 3 months or less.

This still caught me out as on the day the user did change the underlying data the api call was still responding with cached data and would do so until the 8 hours expired. The user experience then meant they’d need to wait until tomorrow before others would see they’d made a change.

This wasn’t ideal. So how do we best satisfy the users with up to date data?

Continue reading
Laravel Native Events — June 11, 2019
Windows, Apache 2.4 and OpenSSL (Revisited) — June 7, 2019
VS Code Extensions — May 31, 2019

VS Code Extensions

I’m an Atom fan and have used it on Linux for ages. I probably still will, but our other developers tend to use VS Code because of their use of MS Windows. I thought I’d revisit VS Code and see if I can use it effectively like Atom.

First thing I needed to make sure of was that some of the Atom features I use are available in VS Code. Standard stuff like dark colour theme, ESLint, syntax highlighting, auto-completions and Emmet.

One pleasant surprise was that VS Code has Emmet built right in! The usage is a little different, keyboard shortcuts etc. but it’s native to VS Code so that’s pretty good.

Other Extensions that I’ve added to support the languages I use are:

Vetur – https://vuejs.github.io/vetur/

ESLint – https://github.com/Microsoft/vscode-eslint

Toggle Excluded Files – https://marketplace.visualstudio.com/items?itemName=eamodio.toggle-excluded-files

Bracket Pair Colorizer – https://marketplace.visualstudio.com/items?itemName=CoenraadS.bracket-pair-colorizer

phpcs – https://github.com/ikappas/vscode-phpcs requires PHP CodeSniffer

PHP DocBlocker – https://github.com/neild3r/vscode-php-docblocker

FiraCode font https://github.com/tonsky/FiraCode

rest-client https://marketplace.visualstudio.com/items?itemName=humao.rest-client

Settings Sync https://marketplace.visualstudio.com/items?itemName=Shan.code-settings-sync

Linux and MS SQL ODBC Authentication — May 16, 2019

Linux and MS SQL ODBC Authentication

My recent troubles spawned from connecting my Laravel application to a Microsoft SQL Server database. My username and password are correct but the connection fails.

In the storage/logs/laravel.log file I found the following:

[2019-05-16 08:22:46] work.ERROR: SQLSTATE[HY000]: [Microsoft][ODBC Driver 17 for SQL Server]SSPI Provider: No Kerberos credentials available (default cache: FILE:/tmp/krb5cc_1000) (SQL: select * from holidays where holiday_date > '2019-05-16 08:22:46') {"exception":"[object] (Illuminate\Database\QueryException(code: HY000): SQLSTATE[HY000]: [Microsoft][ODBC Driver 17 for SQL Server]SSPI Provider: No Kerberos credentials available (default cache: FILE:/tmp/krb5cc_1000) (SQL: select * from holidays where holiday_date > '2019-05-16 08:22:46') at /PATH/vendor/laravel/framework/src/Illuminate/Database/Connection.php:664, Doctrine\DBAL\Driver\PDOException(code: HY000): SQLSTATE[HY000]: [Microsoft][ODBC Driver 17 for SQL Server]SSPI Provider: No Kerberos credentials available (default cache: FILE:/tmp/krb5cc_1000) at /PATH/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php:47, PDOException(code: HY000): SQLSTATE[HY000]: [Microsoft][ODBC Driver 17 for SQL Server]SSPI Provider: No Kerberos credentials available (default cache: FILE:/tmp/krb5cc_1000) at /PATH/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php:43)

I haven’t setup any Kerberos on my Linux side to authenticate and it would appear that my MS SQL wants me to use that. So instead of setting up Kerberos I opted for getting the ODBC connection NOT to use Kerberos.

To do this for my specific user account on my development system I created an ~/.odbc.ini file and added details to disable Kerberos:

.odbc.ini

[my_database]
 Driver = ODBC Driver 17 for SQL Server
 Server = myserver.domain.local,1433
 Database = my_database
# If NOT using Kerberos authentication:
 Trusted_Connection = No
 ServerSPN = MSSQLSvc/myserver.domain.local:my_database
# If using SSL encryption:
 Encryption = Yes
# If using SSL and not importing the server certificate into your certificate store:
 TrustServerCertificate = Yes

To ensure that Laravel makes use of this config the only way I could do it was to NOT use the eloquent models and had to use PHP PDO directly.

// We need to use a direct PDO here because of the
// Trusted Connection=No / SSPI required in .odbc.ini
$pdo = new \PDO('sqlsrv:Server='.env('DB_HOST'),
  env('DB_USER'), env('DB_PASSWORD'));

// Create the number of token parameters
$params = implode(',', array_fill(0, count($tokens), '?'));
$sql = "select data from my_table where something in ($params);";
$stmt = $pdo->prepare($sql);
$stmt->execute($tokens);

$rows = $stmt->fetchAll(\PDO::FETCH_ASSOC); // Associated Array only

ESPAsyncWebServer — March 28, 2019

ESPAsyncWebServer

This is a great ESP8266 component that greatly simplifies the deployment of a web server. It’s very capable and can handle websockets, compressed files and compared to other ESP web servers is streets ahead in terms of performance and abilities – It’s not without it’s challenges though.

One challenge in particular is the use of the ArduinoJson module. Or more correctly the use of ArduinoJson v5, when the most current is v6. As I discovered When I needed the AsyncJson functions, I found that I was using ArduinoJson v6 by default. This prevented my project from compiling because ESPAsyncWebServer is coded using v5 in its AsyncJson module.

A downgrade of ArduinoJson to v5 should have solved the issue, but up popped another issue. Looking through the Github issues history I found exactly my problem, but relating to PlatformIO not Arduino IDE.

error: expected class-name before '{' token
class AsyncJsonResponse: public AsyncAbstractResponse {

https://github.com/me-no-dev/ESPAsyncWebServer/issues/475#issuecomment-464770544

The solution was simply reorder my includes so AsyncJson is after the ESPAsyncWebServer. eg.

#include <ESPAsyncWebServer.h>
#include <AsyncJson.h>
#include <ArduinoJson.h>

The next quirk I encountered was the handler for receiving and processing json in the request body. The example given in the docs:

AsyncCallbackJsonWebHandler* handler = new AsyncCallbackJsonWebHandler("/api", [](AsyncWebServerRequest *request, JsonVariant &json) {
  JsonObject& jsonObj = json.as<JsonObject>();
});
server.addHandler(handler);

Never triggered even when my body had json in it. In the Github issues I found that this is down to the AsyncJson.h code not allowing for a Content-Type header that also included the code page data. There are two choices, modify the AsyncJson.h or set your client to match the exact Content-Type requirement of application/json; and not application/json; charset=UTF-8.

In my case I was able to modify the clients JavaScript axios call to specify the precise header without the code page. If you look at the AsyncJson.h file you’ll find


if (!request->contentType().equalsIgnoreCase(JSON_MIMETYPE))

So there’s no allowance for anything but the specific mime type and no code page. (See jnicolson’s comment)

Hopefully there’ll be an upgrade to ESPAsyncWebServer that moves it to ArduinoJson v6, but right now using v5 is the answer.

References

Scorpion Lite

Scorpion Lite – ESP8266 — March 27, 2019

Scorpion Lite – ESP8266

Looks like my friend likes to try to go one better. When he saw how I’d taken his original Wake-On-Wi-Fi idea and turned it into project Scorpion he moved to using ESP8266.

At the time I started looking into using a Raspberry Pi for the project I did also look at some very lightweight alternatives using the ESP8266 chip set. These are small devices perfect for use in a plethora of IoT projects. Their size is their strength and, for me, their weakness.

Moving onto a small chip means I have a few issues to solve that the Raspberry solves easily. 1) Power supply 2) File storage 3) My skills.

Adafruit Huzzah and FTDI Cable

Something I already had in my box of wires and gadgets is an Adafruit Huzzah ESP8266 board. I bought it a while ago and needed to combine it with a few other bits to make it work. To program it I needed a USB FTDI cable – which I also have.

But then my friend discovered the Wemos D1 Mini – now re-branded Lolin. The big benefit here is that it solves problem #1 (power supply) and means I don’t need an FTDI cable, just a regular micro USB cable which also supplies power.

Wemos also have a relay “shield” that means I can create a tidy package that builds in the ESP8266 and includes the relay required to turn on the PC.

Continue reading
Laravel Echo and Redis — March 18, 2019

Laravel Echo and Redis

The corporate app just got made a bit smarter.

In order to alert users that something has happened that they should be aware of we’ve added real time notifications. We could have polled a table for notifications, but a far smarter development is to use real time communications with socket.io.

Laravel already has these capabilities, but needs to leverage a couple of other services outside of your web server. Notably, Laravel Echo and a message broker such as Pusher or Redis. We chose Redis.

Continue reading
PostGIS and geoJson — February 22, 2019

PostGIS and geoJson

On with this current theme of spacial systems and mapping, one of the interesting challenges I faced was querying spacial data.

My particular problem was trying not to publish masses of polygons up onto an Esri map layer and make the data layer too cumbersome for our intended visitors purpose.

We have a lot of plots of land that we care for and needed to check that when a web site visitor clicks on the Esri map that we check to see if the location they have clicked actually belongs to a plot we manage.

Continue reading
ESRI ArcGIS — February 21, 2019

ESRI ArcGIS

Over the past few weeks I’ve entered into the world of spacial systems and mapping. We have a couple of members of staff who look after our GIS (Geographic Information Systems) that plot all kinds of spacial data onto maps for properties, water courses, streets, etc. The data they produce is used widely through many of our services, but now we’re starting to provide more interactive online mapping for the public.

ESRI ArcGIS Online

Continue reading
Apt Version Pinning — February 7, 2019

Apt Version Pinning

Today after running some apt upgrades my Laravel development environment failed to compile because of a newer version of nodejs than I currently require.

Module build failed: ModuleBuildError: Module build failed: Error: Missing binding /home/paulb/itsm/node_modules/node-sass/vendor/linux-x64-64/binding.node
Node Sass could not find a binding for your current environment: Linux 64-bit with Node.js 10.x

Found bindings for the following environments:
  - Linux 64-bit with Node.js 8.x

This usually happens because your environment has changed since running `npm install`.
Run `npm rebuild node-sass` to download the binding for your current environment.
Continue reading
PHPUnit – Version Mismatch — February 4, 2019

PHPUnit – Version Mismatch

As our codebase matures we return to develop unit tests to ensure our QA process captures any code changes that may have altered the functionality of the product.

When calling PHPUnit on Windows or Linux we ran into some issues relating to the version of PHPUnit we had installed.

On Windows it was an ancient PHPUnit version 3 and on Linux It was running version 7. Neither of which were compatible with our Laravel 5.5 project which uses php version 7.0.

In order to use PHPUnit with our project we must use PHPUnit version 6 (see Supported Versions)

What I hadn’t realised is that we had installed PHPUnit both locally into the OS and with our Laravel project so it exists in composer.json and gets installed under the projects ./vendor. The version installed in the OS path is the version that isn’t compatible with our project, but because it’s in our path it’s taking precedence over our project installed version.

To run the project version we just need to be specific in how we call it.

$ ./vendor/phpunit/phpunit/phpunit
PHPUnit 6.5.13 by Sebastian Bergmann and contributors.

....F                                                               5 / 5 (100%)

Time: 345 ms, Memory: 16.00MB

There was 1 failure:

1) Tests\Unit\Finance\CostCodeTest::testApiGetCostCodes
Expected status code 401 but received 200.
Failed asserting that false is true.

/home/user/itsm/vendor/laravel/framework/src/Illuminate/Foundation/Testing/TestResponse.php:78
/home/user/itsm/tests/Unit/Finance/CostCodeTest.php:37

FAILURES!
Tests: 5, Assertions: 14, Failures: 1.