Stuff I'm Up To

Technical Ramblings

Laravel and O365 Authentication — September 4, 2019

Laravel and O365 Authentication

Our app currently uses LDAP authentication but as our environment is rapidly moving onto the cloud and Microsoft Office 365 it’s time to investigate authentication using O365, more specifically Azure Active Directory.

Getting this going was actually more straight forward than I expected. Laravel already has an authentication provider for OAuth2 called Socialite. Once installed I needed to add in the ‘microsoft-graph’ driver.

The key piece of the puzzle is here: https://socialiteproviders.netlify.com/providers/microsoft-graph.html

Continue reading
Advertisements
PHPUnit Testing Failure — August 14, 2019

PHPUnit Testing Failure

Today’s challenge caused me to burn a lot of time before resolving the issue, which turned out to be an obvious mistake.

When I run a unit test I need to add a number of test rows to the database using the Faker that is built into Laravel. So I added this into my setUp() function.

When the test is finished I need to tidy up and remove the rows I added which I put into my tearDown() function.

The first thing done inside a function override is call the parent function so we call parent::setUp() and parent::tearDown() respectively. Almost all my test are like this. But when I reached one of them it failed and I could not figure out why.

PHPUnit 6.5.14 by Sebastian Bergmann and contributors.

 E                                       1 / 1 (100%)

 Time: 321 ms, Memory: 30.00MB

 There was 1 error:

 1) Tests\Unit\Finance\CostCentreOwnersTest::testApiGetCostCodes
  ReflectionException: Class config does not exist
... 
  /…/app/Observers/CostCentreOwnersObserver.php:78
  /…/app/Observers/CostCentreOwnersObserver.php:69

I refactored a lot of code trying to find this issue. The light bulb moment was seeing only one of my deletions happening in the tearDown(). I then spotted the CostCentreOwnersObserver in the errors.

Of course! I’m using an Eloquent event trigger within the Observer and that is now failing. It’s the only difference to the other tests – they don’t have events.

I realised my simple mistake. I was calling the parent::tearDown() before I was deleting my test rows. So the actual failure was relating to me destroying the environment before I’ve finished with it.

Because the other test models don’t have Eloquent events the parent::tearDown() being in the wrong order didn’t affect them.

TL;DR

The Moral of this story is make sure your call to parent::tearDown() is the last thing in your tests tearDown() function.

Guzzle and Curl — August 12, 2019

Guzzle and Curl

Related to my previous post about Laravel. Guzzle and Nginx I ran into an issue with our proxy. The proxy is always a source of fun and games.

Because the proxy breaks open SSL traffic to scan the content the clients are required to have an SSL certificate installed that tells them to trust our proxy server certificate. In Windows and Linux you can insert the CA cert into the OS using group policy or writing it into the certificate store.

Curl uses it’s own certificate store so we needed to copy the proxy CA cert into the curl store.

On Windows there wasn’t a certificate store. I created one in a location that would remain even if anything was updated or moved.

Download the cacert.pem file and place it in c:\certs. Then I just added my proxy cert in PEM on the end.

C:> type proxy.pem >> c:\certs\cacert.pem

Edit your php.ini and change the curl setting to point at the new cacert.pem file

[curl]
 curl.cainfo = c:/certs/cacert.pem

You can find what php.ini you are using with:

C:> php --ini
Configuration File (php.ini) Path: C:\windows
 Loaded Configuration File:         C:\tools\php73\php.ini
 Scan for additional .ini files in: (none)
 Additional .ini files parsed:      (none)

Restart any php service, like Apache, Nginx, Artisan, etc. and curl should then trust the proxy server.

Laravel, Guzzle and Nginx — August 7, 2019

Laravel, Guzzle and Nginx

After deploying a working test into our pre-production environment the Guzzle API calls we were making to fetch bank holiday data from the .gov.uk site starting failing.

Error creating resource: [message] fopen(https://www.gov.uk/bank-holidays.json): failed to open stream: Connection timed out
 [file] /var/www/itsmpreprod/vendor/guzzlehttp/guzzle/src/Handler/StreamHandler.php
 [line] 323 {"exception":"[object] (GuzzleHttp\Exception\RequestException(code: 0): Error creating resource: [message] fopen(https://www.gov.uk/bank-holidays.json): failed to open stream: Connection timed out

I spent quite some time trying to figure out what the issue was. Nginx would return a 504 Gateway timed out message.

It transpires that Guzzle is pretty smart in what it does. It’s capable of using various types of calls to retrieve the data. In a development environment, under the artisan server, it was happy using tcp sockets, but once on the server under Nginx it tries to use curl.

I got my clue from here:

https://github.com/guzzle/guzzle/issues/1841#issuecomment-341395019

It looks like the issue was just because we hadn’t got the php module php7.0-curl installed!

$ sudo apt install php7.0-curl

Once installed we had to change the proxy scheme from tcp to http and the calls then worked as expected.

    public function getApiData()
    {
        $client = new Client();
        $res = $client->request(
            'GET', 'https://www.gov.uk/bank-holidays.json',
            [
                'proxy' => 'http://proxy:port'
            ]
        );
        return $res->getBody()->getContents();
    }
[unixODBC][Driver Manager]Can’t open lib : file not found — July 12, 2019

[unixODBC][Driver Manager]Can’t open lib : file not found

I have no idea how we came up against this issue on one of the development images. I’d prepared it all up to the point of delivering php. After following my instructions to install the MS SQL drivers everything looked to go well, but when serving up our Laravel project in artisan PHP came up with this error message.

[unixODBC][Driver Manager]Can’t open lib ‘/opt/microsoft/msodbcsql17/lib64/libmsodbcsql-17.3.so.1.1’ : file not found

Now we have seen this before and it related to locale’s so we tried that fix and still didn’t get it to work.

Trawling the internet I came across something pointed us to use ldd to look at the .so file and check out it’s dependencies.

Continue reading
PHP 7.1 and MSSQL Compile Issue — July 8, 2019
Laravel Caching Indefinitely — June 28, 2019

Laravel Caching Indefinitely

I recently attended the Laravel UK conference and learned a lot of very useful things. One of them related to a caching methodology being used for data that rarely changes.

Up until now I’d been taking best guesses to how long to cache data for on a case by case basis. In one particular example I reckoned 8 hours was about right as the data only really changed once every 3 months or less.

This still caught me out as on the day the user did change the underlying data the api call was still responding with cached data and would do so until the 8 hours expired. The user experience then meant they’d need to wait until tomorrow before others would see they’d made a change.

This wasn’t ideal. So how do we best satisfy the users with up to date data?

Continue reading
VS Code Extensions — May 31, 2019

VS Code Extensions

I’m an Atom fan and have used it on Linux for ages. I probably still will, but our other developers tend to use VS Code because of their use of MS Windows. I thought I’d revisit VS Code and see if I can use it effectively like Atom.

First thing I needed to make sure of was that some of the Atom features I use are available in VS Code. Standard stuff like dark colour theme, ESLint, syntax highlighting, auto-completions and Emmet.

One pleasant surprise was that VS Code has Emmet built right in! The usage is a little different, keyboard shortcuts etc. but it’s native to VS Code so that’s pretty good.

Other Extensions that I’ve added to support the languages I use are:

Vetur – https://vuejs.github.io/vetur/

ESLint – https://github.com/Microsoft/vscode-eslint

Toggle Excluded Files – https://marketplace.visualstudio.com/items?itemName=eamodio.toggle-excluded-files

Bracket Pair Colorizer – https://marketplace.visualstudio.com/items?itemName=CoenraadS.bracket-pair-colorizer

phpcs – https://github.com/ikappas/vscode-phpcs requires PHP CodeSniffer

PHP DocBlocker – https://github.com/neild3r/vscode-php-docblocker

FiraCode font https://github.com/tonsky/FiraCode

rest-client https://marketplace.visualstudio.com/items?itemName=humao.rest-client

Settings Sync https://marketplace.visualstudio.com/items?itemName=Shan.code-settings-sync

PHPUnit – Version Mismatch — February 4, 2019

PHPUnit – Version Mismatch

As our codebase matures we return to develop unit tests to ensure our QA process captures any code changes that may have altered the functionality of the product.

When calling PHPUnit on Windows or Linux we ran into some issues relating to the version of PHPUnit we had installed.

On Windows it was an ancient PHPUnit version 3 and on Linux It was running version 7. Neither of which were compatible with our Laravel 5.5 project which uses php version 7.0.

In order to use PHPUnit with our project we must use PHPUnit version 6 (see Supported Versions)

What I hadn’t realised is that we had installed PHPUnit both locally into the OS and with our Laravel project so it exists in composer.json and gets installed under the projects ./vendor. The version installed in the OS path is the version that isn’t compatible with our project, but because it’s in our path it’s taking precedence over our project installed version.

To run the project version we just need to be specific in how we call it.

$ ./vendor/phpunit/phpunit/phpunit
PHPUnit 6.5.13 by Sebastian Bergmann and contributors.

....F                                                               5 / 5 (100%)

Time: 345 ms, Memory: 16.00MB

There was 1 failure:

1) Tests\Unit\Finance\CostCodeTest::testApiGetCostCodes
Expected status code 401 but received 200.
Failed asserting that false is true.

/home/user/itsm/vendor/laravel/framework/src/Illuminate/Foundation/Testing/TestResponse.php:78
/home/user/itsm/tests/Unit/Finance/CostCodeTest.php:37

FAILURES!
Tests: 5, Assertions: 14, Failures: 1.

php 7.0 on Debian Buster — January 2, 2019

php 7.0 on Debian Buster

Actually this is more about any version of php (5.6, 7.0, 7.1, 7.2) on buster. Php source has taken on a bit of a split and the standard repositories only deal with the one supported version for the current release of Debian you are using.

This means that on Debian 9 (buster/sid) the only version available from the Debian repository is php 7.3.

Our current production systems are Debian 9 stretch and only support php 7.0 and therefore only Laravel 5.5. In order to bring my development platform down to php 7.0 I must use a non-standard repository.

Ondřej Surý has been packaging php for Debian and Ubuntu and distributing them. To get them you need to add his key and repository into your aptitude:

$ wget -q https://packages.sury.org/php/apt.gpg -O- | sudo apt-key add -
$ echo "deb https://packages.sury.org/php/ stretch main" | sudo tee /etc/apt/sources.list.d/php.list

Now you can add in whatever version of php you’d like even 5.6. eg.

$ sudo apt-get install php7.0-fpm php7.0-mbstring php7.0-zip php7.0-mysql php7.0-sqlite3 php7.0-dev php-pear

If you already had 7.3 installed nothing will have changed yet and when you type php from the command line you’ll see it still runs version 7.3.

$ php -v

PHP 7.3.0-2+0~20181217092659.24+stretch~1.gbp54e52f (cli) (built: Dec 17 2018 09:26:59) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.3.0-dev, Copyright (c) 1998-2018 Zend Technologies
with Zend OPcache v7.3.0-2+0~20181217092659.24+stretch~1.gbp54e52f, Copyright (c) 1999-2018, by Zend Technologies

To switch back to 7.0 use the following and you’ll see your php go back to 7.0. Switch back in the same way, but replace 7.0 with 7.3.

$ sudo update-alternatives --set php /usr/bin/php7.0

update-alternatives: using /usr/bin/php7.0 to provide /usr/bin/php (php) in manual mode
$ php -v

PHP 7.0.33-1+0~20181208203126.8+stretch~1.gbp2ff763 (cli) (built: Dec 8 2018 20:31:26) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2017 Zend Technologies
with Zend OPcache v7.0.33-1+0~20181208203126.8+stretch~1.gbp2ff763, Copyright (c) 1999-2017, by Zend Technologies

Laravel Routing Returns 302 Redirect — November 20, 2018

Laravel Routing Returns 302 Redirect

That was a frustrating few hours. I added a route into my api.php route and every time I visited it it triggered a redirect. I saw nothing in the browser and it was like my controller function wasn’t even being called.

I put the usual dd() and dump() into my function and it wasn’t being called. I even changed the function name so it didn’t exist and thought I’d get an error message, but nothing – still a redirect.

What is going on? I expanded out the call from “Namespace\Class@function” to put in a function () {} and that was being called and ran ok.

Clearly there was something wrong with my class somewhere. The other functions within it worked fine, just this new one didn’t even seem to be there.

Sure enough a bit of Duck-Jitsu and I found this: https://stackoverflow.com/questions/35020477/laravel-unexpected-redirects-302 – answer 3 is where I got my clue.

I looked at my class constructor and I was using the auth middleware to ensure the functions were not used unless authenticated.

public function __construct() {
  $this->middleware(['auth:api']);
}

For my function I wasn’t using any authentication, so of course I was getting a redirect. But because it’s an api call and the expected return in JSON under an XHttpResponse I wasn’t able to properly debug it. I was getting back a HTML page instead.

As a work around all I needed to do was add in an exception for my function:

public function __construct() {
  $this->middleware(['auth:api'], ['except' => [ 'myFunction' ]]);
}

Now all I have to do is go back and sort out the authentication for the function so I don’t need to bypass it.

When is a Question Mark not a ? — August 29, 2018

When is a Question Mark not a ?

That’s a morning of smashing my face on the desk again. I deployed my dev program onto a production system and then started crying as it stopped working as it should.

It seemed that none of my query string parameters were making it through to the controller. I called up some debugging and dumped out my $request and $request->all() etc. and discovered that the parameters although shown in the browser dev window went AWOL between server and controller. On my dev environment it all acted as it should.

So there must be something different. PHP v7.2 on dev and v7.0 or production maybe? No, much simpler than that. None of the Laracasts and Laravel related Googling pulled up any particular clues. It wasn’t until I looked at Nginx and parameters not being passed to PHP that I got a hit.

https://serverfault.com/questions/685525/nginx-php-fpm-query-parameters-wont-be-passed-to-php

The answer was as simple as adding in the $is_args into my Nginx virtual server config.

location / {
  try_files %uri $uri/ /index.php$is_args$query_string;
}

Up until now I guess I’ve been using routing with the parameters as part of the URI. Now I’m using some query string parameters I need to put in the ?, which is the $is_args variable.

So why not a problem in dev? Because I’m not using Nginx, I just use artisan serve to debug my development program.