FileMaker, cURL and HTTP Response Codes

When FileMaker Inc. released the FileMaker 16 Platform last year they included a feature I had been requesting for over 15 years – native cURL support. cURL is a command line tool and library for transferring data with URLs – you’re probably familiar with the HTTP, HTTPS, FTP and SMTP protocols. cURL is important as it allows us to interact with external web services/APIs directly from FileMaker Pro, FileMaker Go, FileMaker WebDirect and FileMaker Server. For example we can use FileMaker to send an SMS, upload an Invoice to Xero or MYOB, or download Orders from eCommerce platforms like WooCommerce or Shopify.

Starting with the introduction of the Insert From URL script step in FileMaker Pro v12 we’ve been able to interact with URLs with some limitations:

  • FileMaker Pro v12 introduced the Insert From URL script step which supported http, https, ftp, ftps, and file protocols. A field was required to download the result/response into and it didn’t support performing an HTTP/S POST request – only GET requests were supported (most APIs require a POST request to create/update records via the API). There was also no ability to customise the HTTP request, e.g. to set HTTP Headers which are often required for many APIs
  • FileMaker Pro v13 extended the Insert From URL script step to both HTTP GET and HTTP POST requests using the httppost and httpspost custom schemes that you specified when constructing the URL (e.g. “httppost://www.filemaker.com/path?fname=Bob&lname=Smith”). You still couldn’t specify HTTP Headers and still required a field for the result

Given these limitations I rarely used the Insert from URL script step until FileMaker Pro 16 was released. Since the FileMaker Pro v6 days I have been using a number of different plug-ins to perform HTTP requests – initially I had a custom plug-in developed, then switched to the Troi URL plug-in and then to the BaseElements plug-in. Databuzz sponsored the development of some specific functions that we needed and the BaseElements plug-in provided us with cURL support and a number of other related functions, such as JSON encoding and parsing and XPath for parsing XML data. The BaseElements plug-in has provided us with the functionality we needed that was missing in the FileMaker platform, but as a plug-in was required it did have a number of disadvantages:

  • FileMaker Go does not support plug-ins (you can now use the iOS SDK). You could use FileMaker Server script schedules and Perform Script on Server as a workaround but this required the FileMaker solution to be hosted by FileMaker Server so wouldn’t work for solutions running locally on the iPad or iPhone
  • running scripts under FileMaker Server required the plug-in to be installed on the server, which can be done easily as long as the appropriate permissions have been granted in the FileMaker Server Admin Console. Some hosting providers also disabled the option to install plug-ins on shared servers.

With the release of the FileMaker 16 platform came the ability to specify one or more supported cURL options as a calculation when using the Insert From URL script step, as well as the ability to specify a variable as the target (no more fields required!). This was a game changer as far as working with APIs was concerned – now the entire FileMaker Platform could natively make HTTP requests, set HTTP Headers, and encode JSON data and parse JSON. All of our integration solutions were using the BaseElements plug-in, so we started to work on adding native support for FileMaker Pro v16 users by switching from the BaseElements plug-in functions to using the native Insert From URL script step, cURL options and JSON functions.

One function for which there was no native equivalent was the BE_HTTP_Response_Code function which returns the HTTP response code value from the last request. HTTP Response Codes are issued by the server in response to a request and can be used to determine whether the HTTP request was successfully completed or if there were issues associated with the request. For example 200 is the standard response for a successful request and 401 means there was an authentication issue and the request failed. It’s important when working with APIs to check the HTTP Response Code to confirm that the request was processed successfully – the documentation for each API will usually specify what response code indicates a successful request (typically 200 or 201).

The HTTP Response Code is contained with the HTTP Response Headers which are returned by the server processing the request. You need to specify the following cURL option to get the response headers returned into a FileMaker variable:

" --dump-header $responseHeaders"

The $responseHeaders variable will then contain something like this after the Insert From URL script step has been performed:

HTTP/1.1 200 OK
Date: Wed, 13 Jun 2018 04:45:17 GMT
Server: Apache/2.4.26 (Red Hat)
X-Powered-By: PHP/5.6.35
Transfer-Encoding: chunked
Content-Type: application/json; charset=UTF-8

In the above example you can see the first line of the response headers contains the response code: “HTTP/1.1 200 OK”. Other common response codes that you might encounter include:

HTTP/1.1 100 Continue
HTTP/1.1 403 Forbidden
HTTP/1.1 201 Created
HTTP/1.1 302 Found
HTTP/1.1 301 Moved Permanently

You can use standard FileMaker functions to get the actual numerical response code from the first line of the response headers, e.g.:

Let ( [
t1 = GetValue ( $responseHeaders ; 1 ) ;
t2 = Substitute ( t1 ; " " ; "¶" ) ;
n1 = GetValue ( t2 ; 2 )
] ;

n1

) // Let

This all works fine as long as the Response Headers only contain a single response code, but sometimes the server will return multiple response codes. For example the response might include a redirect or a continue and look like this:

HTTP/1.1 100 Continue

HTTP/1.1 403 Forbidden
Server: nginx/1.14.0
Date: Wed, 13 Jun 2018 07:41:16 GMT
Connection: keep-alive

The response code we are after is the “403” value and not the “100” value, so our above calculation would return the wrong response code in this situation. We started working on a custom function that would look for the last “HTTP/1.1” string and then get the code following this, but that failed pretty quickly when working with a server that included the string “HTTP/1.1” elsewhere in the response headers.

We’ve settled on using this custom function which so far is returning the correct result, at least with the APIs and servers that we have tested it on so far. We would much rather have a native function for this which does the equivalent of what the BaseElements BE_HTTP_Response_Code function does without having to try to locate and find the last response code from the response headers. From researching this it appears that there is already a cURL option for this:

-w "%{http_code}"

but unfortunately the -w option is not one of the FileMaker supported cURL options, so for now you’ll need to parse out the response code until FileMaker add support for the -w option. We’ve added this to the FileMaker Product Ideas discussion area – if you would also like to see this feature in a future version of FileMaker platform you can vote up the idea here.

FileMaker Inc. releases FileMaker 17 Platform

FileMaker Inc. today released the FileMaker 17 Platform with new versions of FileMaker Pro Advanced, Server, FileMaker Go and FileMaker WebDirect. As with previous releases FileMaker 17 has a number of new features for both customers and developers alike. This releases marks the fourth annual release that started with v14 with all new versions shipping in May each year.

Updates in the FileMaker 17 Platform include:

Development:

  • Starter apps: Get started more quickly by choosing one of six new starter apps. Add more functionality to an app by connecting an add-on table.
  • Master-detail layouts: Leverage the new portal enhancements to create common design patterns like master-detail layouts.
  • Redesigned layout mode: Create custom apps more easily with layout tools that are more discoverable through convenient panes inside the document window.

Mobility:

  • Sensor support: Using a new calculation function, mobile apps can capture information from iOS sensors in iPad or iPhone devices.
  • Configure local notification: Display a local notification on an iPad or iPhone device when FileMaker Go is not running or is in the background.
  • Drag and drop:  Go faster with drag and drop of text, photos, and files between apps on iPad devices running iOS 11.2.

Administration and Integration:

  • FileMaker Server Admin Console: Redesigned to be more lightweight with a streamlined user interface.
  • FileMaker Admin API trial: Get REST API access to manage and administer FileMaker Server. Trial expires Sept. 27, 2019.
  • FileMaker Data API: This improved REST API includes support for FileMaker Server scripts, the ability to upload files to container fields, and a more standardized API format.

The release of the FileMaker 17 Platform also sees the end of FileMaker Pro as a standalone application, as it is now replaced by FileMaker Pro Advanced. FileMaker Pro and FileMaker Pro Advanced are no longer sold as separate products – you now enable the Advanced developer features such as the Script Debugger and Data Viewer when the Use advanced tools general preference is selected.

FileMaker Inc. also released a new FileMaker data migration tool that can significantly reduce the time it takes to migrate/import data from one version of a FileMaker solution to another (e.g. when moving data from a production version to the latest development version). The data migration process can now go from days to hours or hours to minutes when importing large data sets. This new time-saving command-line tool helps you update your deployed custom apps in no time. The data migration tool is available through the FileMaker Developer Subscription.

With the  launch of the FileMaker 17 Platform FileMaker Inc. have also announced new licensing programs that replace the previous programs – the primary new offering is called FileMaker User Licensing. FileMaker User Licensing is a simple, cost-effective way for teams to license the entire FileMaker Platform. FileMaker User Licensing is based on the number of unique users a company or organization has that need to use FileMaker software. The key components of the new FileMaker User Licensing Program are:

  1. Every license includes the entire FileMaker 17 Platform – FileMaker Server, FileMaker Go, FileMaker WebDirect, and FileMaker Pro Advanced – which replaces FileMaker Pro – to give everyone access to advanced development and customization features.
  2. Start at 5, add in increments of 1 – the initial minimum quantity is 5 users, and you can add users in increments of 1.
  3. New FileMaker Data API – each license comes with unlimited inbound data transfer and 2 GBs of outbound data transfer, per user, per month – shared and tracked annually.
  4. All new contracts are managed under one license key – and the license key will not change over time making renewals and additions much easier to manage.

We’re please to report that all Databuzz products are compatible with the FileMaker 17 Platform:

We’ll be publishing more articles over the coming months looking at some of the new features in the FileMaker 17 Platform in more depth – you can subscribe to our newsletter to be notified when they are released or follow us on Twitter or Facebook.

FileMaker 17 Platform Compatibility Update

The FileMaker 17 Platform was released today and we’re pleased to report that all Databuzz products are compatible with the FileMaker 17 Platform. We haven’t encountered any issues with the following products in our testing with the FileMaker 17 Platform:

fmSMS – we have sent messages from multiple SMS Gateways, checked Account balances and checked the status of sent Messages successfully.

fmAccounting Link (Xero Edition) – we have been able to authenticate, download from Xero to FileMaker and upload from FileMaker to Xero successfully

fmEcommerce Link (WooCommerce Edition) – we have been able to authenticate, download from WooCommerce to FileMaker and upload from FileMaker to WooCommerce successfully

fmEcommerce Link (Shopify Edition) – we have been able to authenticate, download from Shopify to FileMaker and upload from FileMaker to Shopify successfully

fmAccounting Link (MYOB AccountRight Edition) – we have been able to authenticate, download from AccountRight to FileMaker and upload from FileMaker to AccountRight successfully

fmAccounting Link (MYOB Essentials Edition) – we have been able to authenticate, download from MYOB Essentials to FileMaker and upload from FileMaker to MYOB Essentials successfully

If you encounter any issues with any of our products and FileMaker 17 Platform please let us know. We’ll be writing more articles about some of the new features in the FileMaker 17 Platform over the coming weeks – you can subscribe to our newsletter to be notified when they are released or follow us on Twitter or Facebook

fmEcommerce Link (WooCommerce Edition) Now Runs Natively on FileMaker 16 Platform

When we first released fmEcommerce Link (WooCommerce Edition) back in 2016 the functionality required to integrate with the WooCommerce REST API did not exist in the current version of FileMaker Pro, therefore we had to use a FileMaker Pro plug-in to handle the API requests, set HTTP Headers, encode JSON and parse the JSON response. This allowed fmEcommerce Link (WooCommerce Edition) to run under FileMaker Pro v12 and later, as well as under FileMaker Server with the plug-in installed, however it prevented it from running on FileMaker Go which does not currently support plug-ins (except via the iOS App SDK).

With the release of the FileMaker 16 platform we got a number of new functions and script steps, including support for working with JSON data and being able to specify supported cURL options with the Insert From URL script step. These new features meant that we were no longer dependent on plug-in functions and we could replace these with native FileMaker features. We’re pleased to announce that v1.5 of fmEcommerce Link (WooCommerce Edition) is now available and includes a number of new features, including native support for FileMaker Pro v16 and later. This is a free update to all existing customers (simply download using the same link that was on your original order email).

We’ve updated all the requests for downloading and uploading to the WooCommerce API to use native FileMaker functions and script steps so you no longer need to use a FileMaker plug-in if you’re using FileMaker Pro v16 or higher. This means you can also use fmEcommerce Link on FileMaker Go natively (requires FileMaker Go 16) – we’ve also included an example for using Perform Script on Server to have FileMaker Server handle uploading a Product to WooCommerce.

In working on this update (which was a much bigger update than we anticipated and took much longer than we planned!) we tried to avoid making too many unnecessary schema changes. We’ve simply updated the scripts to check which version of FileMaker Pro, FileMaker Go or FileMaker Server is being used and if you’re using v15 or earlier we simply use the previous plug-in functionality, and if you’re using v16 or later we now use native functionality. Everything works exactly the same regardless of which version of FileMaker Pro you’re using, but we can now include support for FileMaker Go as well as having one less dependency to worry about.

You can get all the details on our version history page – we’re busy working on adding native FileMaker 16 support to our other products, as well as some further updates to fmEcommerce Link (WooCommerce Edition) including support for uploading Orders from FileMaker to WooCommerce.

fmAccounting Link (Xero Edition) and the Xero API TLS 1.0 Deprecation

Xero have recently announced that they will be deprecating support for TLS (Transport Layer Security) 1.0 from 30 June 2018. TLS is used to secure communication between partner apps like fmAccounting Link (Xero Edition) and the Xero API. The Xero API will now require partners apps to use TLS 1.1 as a minimum and recommend upgrading to TLS 1.2.

Xero have recently made available a test URL for partner app developers to test their apps to ensure they will continue to work after 30 June 2018. Databuzz is pleased to report that after performing a series of tests using fmAccounting Link (Xero Edition) with the test URL we encountered no issues and were able to authenticate the private application, request data from Xero and upload data to Xero successfully.

fmAccounting Link (Xero Edition) is also already using TLS 1.2 which will most likely become the mandatory minimum version at some point in the future – many of the other APIs that we work with at Databuzz have already made TLS 1.2 the minimum version.

If you have any questions about fmAccounting Link (Xero Edition) and the Xero API TLS 1.0 Deprecation please contact us.

FM Starting Point Custom Integrations

Over the past few years we’ve performed a number of customisations for customers using the popular FM Starting Point template from Richard Carlton Consulting. FM Starting Point is a free FileMaker CRM template designed for use with the latest version of FileMaker Pro, and is focused on small businesses, work groups, and non-profit organisations. FM Starting Point manages business contacts , accounts, products, projects, and more – since it’s first release 5 years ago it has been downloaded over 490,000 times.

Over the past 5 years we’ve performed the following customisations of FM Starting Point for our customers:

  • added the ability to send single/bulk SMS messages and receive incoming SMS messages (fmSMS integration)
  • integrated our fmAccounting Link (Xero Edition) solution so you can upload Contacts, Invoices, Bills, Payments and more from FM Starting Point to Xero
  • added the ability to send single and bulk HTML emails
  • integrated our fmEcommerce Link (WooCommerce Edition) solution so you can push Products from FM Starting Point to WooCommerce and download Orders from WooCommerce to FM Starting Point
  • integrated our fmEcommerce Link (Shopify Edition) solution so you can push Products from FM Starting Point to Shopify and download Orders from Shopify to FM Starting Point
  • integrated our fmAccounting Link (MYOB AccountRight Edition) solution so you can upload Contacts, Invoices, Bills, Payments and more from FM Starting Point to MYOB AccountRight

If you would like to discuss customising your version of FM Starting Point please get in touch for a free initial consultation to discuss your requirements.

 

fmAccounting Link (MYOB AccountRight Edition) and AccountRight 2018.1.1

We’ve been testing fmAccounting Link (MYOB AccountRight Edition) with AccountRight 2018.1.1 that was released this week and have not found any issues in our testing so far. We’ve been able to download data from AccountRight and upload data to AccountRight successfully so we don’t anticipate any issues with AccountRight 2018.1.1, but please let us know if you encounter any issues when working with AccountRight 2018.1.1.

MYOB have also recently added a new beta API endpoint for Quotes that will allow you to read, create, update and delete sales quotes via the API. This will work in a similar way to Invoices and Orders, and will be available for all quote layouts including Item, Miscellaneous, Professional, Service and Time Billing. Once the Quotes API endpoint is out of beta and available to all users we will add examples for working with Quotes to the fmAccounting Link (MYOB AccountRight Edition) solution as a free update for all existing customers.

Gravity Forms Integration with FileMaker

Gravity Forms is a popular Form plugin for the WordPress platform, allowing you to create simple or advanced forms to capture data on your WordPress website. Over the years we’ve helped many customers with retrieving data from their Gravity Forms using a variety of methods, including:

  • simple CSV file download from Gravity Forms then import that CSV file into FileMaker
  • using the FileMaker PHP API to have Gravity Forms data automatically ‘pushed’ to their FileMaker solution each time an entry is submitted
  • using the External SQL Data Source feature to view the Gravity Forms data live from within FileMaker and create FileMaker scripts to download data

Each of these methods have their pros and cons: for some users the FileMaker PHP API integration is not an option, and for others they aren’t able to connect remotely to the SQL database as this is blocked by their web host. Downloading CSV files each day/week is also time consuming and cumbersome. Fortunately there is another method to use that has many advantages over these methods: the Gravity Forms Web API.

Using the Gravity Forms Web API we can dynamically query Gravity Forms from within FileMaker to check for new form entries and download those directly into our FileMaker solution, all within a few seconds. If you’re using FileMaker Pro v16 you can also use the new native JSON and cURL features to do this without requiring any FileMaker plug-ins – for older versions of FileMaker a plug-in is required to handle the API requests and cryptographic requirements. You can also setup schedules using FileMaker Server to have new form entries downloaded automatically (hourly, daily etc).

Here’s a short video demonstrating how we can download Gravity Forms entries directly into a FileMaker solution (you can also view it directly on YouTube):

If you would like to discuss integrating Gravity Forms with your FileMaker solution please get in touch for a free initial consultation to discuss your requirements.

fmEcommerce Link (Shopify Edition) and Metafields

Metafields in Shopify are like custom fields – they allow you to attach additional metadata or information about a particular Shopify store resource, such as a Product or Product Variant. For example you might need to store information about washing instructions for clothing that you sell or you would like to store a manufacturer and supplier SKU for each Product Variant.

In our fmEcommerce Link (Shopify Edition) solution we support the downloading and uploading of Metafields for Products and Product Variants (the Shopify API allows you to attach Metafields to a number of resources – see here for the full list). Shopify unfortunately does not have an interface for store owners to view or edit the Metafields for your store – it is considered a developer/designer feature so you will need to use a Shopify App to create, edit or delete Metafields in the Shopify Admin page.

Here’s some screenshots showing the Metafields for a Product:

and a Product Variant:

Using fmEcommerce Link (Shopify Edition) you can add, edit and delete these Metafields and using a Shopify app you can also view them. Here’s a screenshot showing the Metafields using the Metafields Editor app:

Here’s some links for learning more about working with Shopify Metafields:

If you would like us to add support for Metafields elsewhere in the fmEcommerce Link (Shopify Edition) solution please let us know.

fmAccounting Link (MYOB AccountRight Edition) Updated for Bills

We’ve just released an update to fmAccounting Link (MYOB AccountRight Edition), our FileMaker solution for integrating with MYOB AccountRight in Australia and New Zealand, with examples for working with Bills in MYOB. With v1.25 of fmAccounting Link you can now use FileMaker to:

  • download Bills from AccountRight to FileMaker (either all Bills or filtered by Bill Date)
  • create Bills in FileMaker and upload them to MYOB
  • convert a Purchase Order to a Bill in FileMaker and upload the converted Bill to MYOB

This is a free update for all existing customers – you can view the full release notes on our version history page.