Overview of curl command?

The curl command, which is used to transport data to or from a server using a variety of protocols like HTTP, HTTPS, FTP, and more, is a commonly used utility in Unix-based systems (including Linux and macOS) and is also available for Windows. It stands for “Client URL” and is very helpful when downloading files, testing APIs, and interacting with web services. The most basic operation of curl is to fetch the content of a URL.

Importance of file downloads automation

Automation of file downloads has many advantages. It saves significant time. Users can concentrate on more important things by having a script manage the work of downloading files instead of having to do it by hand.

By scheduling downloads to happen at predetermined periods, automation systems help guarantee that data is constantly updated.

Human error is a risk with manual downloads, as wrong files may be downloaded, or all necessary files may not be downloaded. Error risk is lower, and accuracy is guaranteed by automation.

By reducing the amount of time that people spend handling sensitive data, automating downloads can improve security. Scripts can manage encryption and authentication securely, lowering the possibility of exposure to unauthorized individuals.

I’ll demonstrate a batch script that downloads builds automatically from a server in this article. To run the batch file, I will also display the customized robot framework keyword.

Environment Setup

• Curl Installation and verification:  Curl is already installed in Windows 10 and later version, Verify curl installation using the version check command.

If curl is not installed, please install it manually.

• Curl download package site link: https://curl.se/download.html, Download the appropriate zip file from the Windows section as per the operating system. https://curl.se/windows/
• Extract the zip file in a folder, say ‘C:\curl’
• Add curl to system path, Edit the path variable and add C:\curl\bin
• Verify the installation with a version check.  Curl –version

Batch file creation.

A batch file is a text file that contains several commands to be run by Windows’ command-line interpreter. These files, which usually have the.bat or.cmd file extension, are used to run intricate command sequences, configure environments, and automate repetitive activities. Batch files are compatible with all versions of Windows, from DOS to modern Windows versions.

Below is a batch file to download from artifactory and move the file to the respective folder. The script will also check the size of the file and move it if the size is greater than 20MB. The below filename is download.bat

set URL=%1
curl -v -m 240 -u <username:password/access_token> -X GET %URL% -H 'Content-Type:application/json' -o app-release.apk
if "%errorlevel%"=="0" (
   echo "Download successful"
   set flag=1
)
for %%A in (app-release.apk) do set BYTES=%%~zA
echo %BYTES%
if %flag%==1 if BYTES gtr 20000000 move app-release.apk .\\App_file\mobile.apk

Explanation of the syntax.

• The user who uses the file will provide the build location. The first line is: set URL=%1 This command is setting the first command line argument to the variable URL.
• The second line is the curl command to download the build from the specified URL.
  • The command is taking the user:password/access_token  for authorization purpose
  • -v produce the output in verbose mode
  • -m is the maximum timeout. The curl command will wait 4 minutes for any download issues. After that, it exits and checks next command
  • -o is writing the downloaded file as the name
  • -O can be used to automatically write output to a file named as the remote file
• The third line is an If condition, which is just checking the execution status of the curl command. If the status is success, then it’s set a flag variable.
• The next line checks the size of the file in bytes.
• The last line is moving the file to the ‘APP_File’ folder if the download is successful and downloaded file size is more than 20MB.

The usage of the curl command.

Execution and scheduling

• The file executed directly from command line, passing the URL value. # download.bat URL
• The file execution can be configured with a CI/CD tool like Jenkins before the actual execution, so that there is no need to download the build manually.
• The file can be configured with windows task scheduler to run it automatically with some period intervals.
• The file can be called in the script so that the build is downloaded and installed on the system before the test.

Below is the sample Robot Framework keyword which runs the process to execute the bat file.

*** Settings ***
Library           Process

*** Variables ***
${PROGRAM_BUILD_DOWNLOAD}        ${CURDIR}\\..\\download.bat ${URL}

*** Keywords ***
BUILD DOWNLOAD
    [Documentation]    Download the test build from the specified URL
    ${result}=    Run Process       ${PROGRAM_BUILD_DOWNLOAD}   timeout=4min     shell=True   on_timeout=continue   alias=Download
    Log    ${result}
    ${result1}    Get Process Result    Download    rc=True    stdout=True    stderr=True

• The run process will start the batch file with a maximum timeout of 4 minutes and will continue execution after the timeout. The process starts with the name ‘Download’.
• The last line checks the process execution result.

If you face any issue writing robot keyword, you can write a sample python function and develop a custom keyword in your robot framework file to use the python function.  For details, please go through https://dasfascination.com/custom-keywords-development-robot-framework/