chenyuxuan/arcs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: migrate project into clean repository

Browse Source
This commit is contained in:
yuuux
2026-08-13 16:50:52 +08:00
commit d1d25a09e7
27405 changed files with 9422808 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

45
arcs-sdk/modules/coreHTTP/CHANGELOG.md Normal file
View File

@@ -0,0 +1,45 @@
# Changelog for coreHTTP Client Library
## v2.1.0 (Nov 2021)
### Updates
- [#114](https://github.com/FreeRTOS/coreHTTP/pull/114) Update http-parser version in manifest to reflect commit
- [#112](https://github.com/FreeRTOS/coreHTTP/pull/112) Add function prototypes for exported functions to CBMC proof harnesses
- [#111](https://github.com/FreeRTOS/coreHTTP/pull/111) Update Doxygen version to 1.9.2
## v2.0.2 (July 2021)
### Updates
- [#109](https://github.com/FreeRTOS/coreHTTP/pull/109) Add C++ header guards
- [#106](https://github.com/FreeRTOS/coreHTTP/pull/106) Update case-insensitive compare function for header-field parser
- [#104](https://github.com/FreeRTOS/coreHTTP/pull/104) Update CBMC proofs to work with the latest version of CBMC
## v2.0.1 (February 2021)
### Other
- [#89](https://github.com/FreeRTOS/coreHTTP/pull/89) Fix documentation of memory size estimates of the library.
## v2.0.0 (December 2020)
### Updates
- [#83](https://github.com/FreeRTOS/coreHTTP/pull/83) Implement transport send and receive retry timeouts in coreHTTP. This change adds a timestamp callback function to the HTTPResponse_t struct, and new configuration macros to set the transport send and receive retry timeouts. Due to the HTTPResponse_t struct field addition, coreHTTP v2.0.0 is not backward compatible under certain conditions.
- [#79](https://github.com/FreeRTOS/coreHTTP/pull/79), [#82](https://github.com/FreeRTOS/coreHTTP/pull/82) transport_interface.h documentation updates.
- [#75](https://github.com/FreeRTOS/coreHTTP/pull/75) Small fix to cast logging arguments to types matching the format specifiers.
### Other
- [#70](https://github.com/FreeRTOS/coreHTTP/pull/70), [#72](https://github.com/FreeRTOS/coreHTTP/pull/72), [#78](https://github.com/FreeRTOS/coreHTTP/pull/78) Github actions updates.
- [#73](https://github.com/FreeRTOS/coreHTTP/pull/73), [#76](https://github.com/FreeRTOS/coreHTTP/pull/76) Github repo chores.
- [#71](https://github.com/FreeRTOS/coreHTTP/pull/71) CBMC automation chore.
- [#81](https://github.com/FreeRTOS/coreHTTP/pull/81), [#84](https://github.com/FreeRTOS/coreHTTP/pull/84) Doxygen memory estimates table update.
## v1.0.0 November 2020
This is the first release of the coreHTTP client library in this repository.
The HTTP client library is a client-side implementation that supports a subset
of the HTTP/1.1 protocol. It is optimized for resource-constrained devices, and
does not allocate any memory.

36
arcs-sdk/modules/coreHTTP/CMakeLists.txt Normal file
View File

@@ -0,0 +1,36 @@
if(DEFINED CONFIG_SDK_MODULE_COREHTTP)
listenai_library_named(coreHTTP)
include_directories(../../lisa_porting/include/lisa_os)
add_definitions(-DHTTP_DO_NOT_USE_CUSTOM_CONFIG)
add_definitions(-Dhttp_message_needs_eof=ls_http_message_needs_eof)
add_definitions(-Dhttp_should_keep_alive=ls_http_should_keep_alive)
add_definitions(-Dhttp_parser_execute=ls_http_parser_execute)
add_definitions(-Dhttp_method_str=ls_http_method_str)
add_definitions(-Dhttp_parser_init=ls_http_parser_init)
add_definitions(-Dhttp_errno_name=ls_http_errno_name)
add_definitions(-Dhttp_errno_description=ls_http_errno_description)
add_definitions(-Dhttp_parser_parse_url=ls_http_parser_parse_url)
add_definitions(-Dhttp_parser_pause=ls_http_parser_pause)
add_definitions(-Dhttp_body_is_final=ls_http_body_is_final)
add_definitions(-Dhttp_parser_set_max_header_size=ls_http_parser_set_max_header_size)
add_definitions(-Dhttp_parser_url_init=ls_http_parser_url_init)
add_definitions(-Dhttp_errno_description=ls_http_errno_description)
add_definitions(-Dhttp_errno_name=ls_http_errno_name)
add_definitions(-Dhttp_status_str=ls_http_status_str)
add_definitions(-Dhttp_parser_settings_init=ls_http_parser_settings_init)
add_definitions(-Dhttp_parser_version=ls_http_parser_version)
listenai_library_sources(
source/core_http_client.c
source/dependency/3rdparty/http_parser/http_parser.c
)
listenai_include_directories(
source/include
source/interface
source/dependency/3rdparty/http_parser
)
endif()

7
arcs-sdk/modules/coreHTTP/Kconfig Normal file
View File

@@ -0,0 +1,7 @@
menu "coreHttp"
config SDK_MODULE_COREHTTP
bool "enable sdk module core http"
default n
endmenu

19
arcs-sdk/modules/coreHTTP/LICENSE Normal file
View File

@@ -0,0 +1,19 @@
MIT License
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

34
arcs-sdk/modules/coreHTTP/MISRA.md Normal file
View File

@@ -0,0 +1,34 @@
# MISRA Compliance
The HTTP Client library files conform to the [MISRA C:2012](https://www.misra.org.uk)
guidelines, with some noted exceptions. Compliance is checked with Coverity static analysis.
Deviations from the MISRA standard are listed below. The deviations below do not
include the third-party [http-parser source code](https://github.com/nodejs/http-parser/tree/v2.9.3):
### Ignored by [Coverity Configuration](https://github.com/aws/aws-iot-device-sdk-embedded-C/blob/main/tools/coverity/misra.config)
| Deviation | Category | Justification |
| :-: | :-: | :-- |
| Directive 4.5 | Advisory | Allow names that MISRA considers ambiguous (such as LogInfo and LogError). |
| Directive 4.8 | Advisory | Allow inclusion of unused types. Header files for a specific port, which are needed by all files, may define types that are not used by a specific file. |
| Directive 4.9 | Advisory | Allow inclusion of function like macros. The `assert` macro is used throughout the library for parameter validation, and logging is done using function like macros. |
| Rule 2.4 | Advisory | Allow unused tags. Some compilers warn if types are not tagged. |
| Rule 2.5 | Advisory | Allow unused macros. Library headers may define macros intended for the application's use, but are not used by a specific file. |
| Rule 3.1 | Required | Allow nested comments. C++ style `//` comments are used in example code within Doxygen documentation blocks. |
| Rule 11.5 | Advisory | Allow casts from `void *`. The third-party http-parser library callback contexts are saved as `void *` and must be cast to the correct data type before use. |
### Flagged by Coverity
| Deviation | Category | Justification |
| :-: | :-: | :-- |
| Directive 4.6 | Advisory | The third-party http-parser library does not use specific-length typedefs for their callback function signatures and public structure fields. http-parser callbacks are implemented in the HTTP Client source and also flags of basic numerical types are checked from the http-parser structure.
| Rule 8.7 | Advisory | API functions are not used by the library outside of the files they are defined; however, they must be externally visible in order to be used by an application. |
| Rule 8.13 | Advisory | The third-party http-parser library callback definitions have a non-const parameter which holds the state of the parsing. This parameter is never updated in the callback implementations, but have fields that may be read. |
| Rule 10.5 | Advisory | The third-party http-parser library has a structure with a field of type unsigned int whose values are intended to be mapped to an enum. This field contains error codes used by the HTTP client library. |
| Rule 14.3 | Required | The third-party http-parser library sets a uint64_t type field to `ULLONG_MAX` or `( ( uint64_t ) -1 )`, during its internal parsing. Coverity MISRA does not detect that this variable changes. This field is checked by the HTTP Client library. |
### Suppressed with Coverity Comments
| Deviation | Category | Justification |
| :-: | :-: | :-- |
| Rule 5.4 | Required | The length of string literal macro identifiers are labeled with the identifier of the string literal itself postfixed with "_LEN" for clarity. This is consistent throughout the library. |
| Rule 10.8 | Required | The size of the headers is found by taking the current location being parsed and subtracting it from the start of the headers. The start of the headers is set on the first header field found from http-parser. This always comes before finding the header length; if it does not, an assertion is triggered. |
| Rule 11.8 | Required | If the response body uses chunked transfer encoding, then it is necessary to copy over the chunk headers with the data to which the current parsing location points. The current parsing location is returned to the callback implementation as a `const char *`. |
| Rule 18.3 | Required | It is expected that the current location http-parser passes into the body parsing callback points to the same user response buffer; if it does not, an assertion is triggered. |

45
arcs-sdk/modules/coreHTTP/Makefile Normal file
View File

@@ -0,0 +1,45 @@
TOPDIR = ../../..
#lib or target you want to build: eg. LIB=libmisc.a or TARGET=wlan
LIB = libcoreHTTP.a
TARGET =
LDS =
#libraries depend: eg. LIBS = -lbsp -ldrv -lrtos
LIBS =
#exclude subdirs
EXDIRS =
#files want to build: eg. CSRCS=x.c SSRCS=x.S
CSRCS = $(shell find . -maxdepth 10 -type f -name "*.c" | grep -v test)
SSRCS =
# OPTIM = -Os
#includings and flags
CFLAGS += -Dhttp_message_needs_eof=ls_http_message_needs_eof
CFLAGS += -Dhttp_should_keep_alive=ls_http_should_keep_alive
CFLAGS += -Dhttp_parser_execute=ls_http_parser_execute
CFLAGS += -Dhttp_method_str=ls_http_method_str
CFLAGS += -Dhttp_parser_init=ls_http_parser_init
CFLAGS += -Dhttp_errno_name=ls_http_errno_name
CFLAGS += -Dhttp_errno_description=ls_http_errno_description
CFLAGS += -Dhttp_parser_parse_url=ls_http_parser_parse_url
CFLAGS += -Dhttp_parser_pause=ls_http_parser_pause
CFLAGS += -Dhttp_body_is_final=ls_http_body_is_final
CFLAGS += -Dhttp_parser_set_max_header_size=ls_http_parser_set_max_header_size
CFLAGS += -Dhttp_parser_url_init=ls_http_parser_url_init
CFLAGS += -Dhttp_errno_description=ls_http_errno_description
CFLAGS += -Dhttp_errno_name=ls_http_errno_name
CFLAGS += -Dhttp_status_str=ls_http_status_str
CFLAGS += -Dhttp_parser_settings_init=ls_http_parser_settings_init
CFLAGS += -Dhttp_parser_version=ls_http_parser_version
CFLAGS += -DHTTP_DO_NOT_USE_CUSTOM_CONFIG
CFLAGS += -I./source/include
CFLAGS += -I./source/interface
CFLAGS += -I./source/dependency/3rdparty/http_parser
CFLAGS += -I$(TOPDIR)/include/lisa_porting/lisa_os
CFLAGS += -I$(TOPDIR)/include/lisa_porting/lisa_log \
$(shell find $(TOPDIR)/include $(TOPDIR)/src -type d -exec echo "-I{}/ " \;)
include $(TOPDIR)/rules.mk

107
arcs-sdk/modules/coreHTTP/README.md Normal file
View File

@@ -0,0 +1,107 @@
# coreHTTP Client Library
This repository contains a C language HTTP client library designed for embedded
platforms. It has no dependencies on any additional libraries other than the
standard C library, [http-parser](https://github.com/nodejs/http-parser), and
a customer-implemented transport interface. This library is distributed under
the [MIT Open Source License](LICENSE).
This library has gone through code quality checks including verification that no
function has a [GNU Complexity](https://www.gnu.org/software/complexity/manual/complexity.html)
score over 8. This library has also undergone both static code analysis from
[Coverity static analysis](https://scan.coverity.com/), and validation of memory
safety and data structure invariance through the
[CBMC automated reasoning tool](https://www.cprover.org/cbmc/).
See memory requirements for this library [here](./docs/doxygen/include/size_table.md).
**coreHTTP v2.0.0 [source code](https://github.com/FreeRTOS/coreHTTP/tree/v2.0.0/source) is part of the [FreeRTOS 202012.00 LTS](https://github.com/FreeRTOS/FreeRTOS-LTS/tree/202012.00-LTS) release.**
## coreHTTP Config File
The HTTP client library exposes configuration macros that are required for
building the library. A list of all the configurations and their default values
are defined in [core_http_config_defaults.h](source/include/core_http_config_defaults.h).
To provide custom values for the configuration macros, a custom config file
named `core_http_config.h` can be provided by the user application to the library.
By default, a `core_http_config.h` custom config is required to build the
library. To disable this requirement and build the library with default
configuration values, provide `HTTP_DO_NOT_USE_CUSTOM_CONFIG` as a compile time
preprocessor macro.
**The HTTP client library can be built by either**:
* Defining a `core_http_config.h` file in the application, and adding it to the
include directories for the library build.
**OR**
* Defining the `HTTP_DO_NOT_USE_CUSTOM_CONFIG` preprocessor macro for the
library build.
## Building the Library
The [httpFilePaths.cmake](httpFilePaths.cmake) file contains the information of
all source files and header include paths required to build the HTTP client
library.
As mentioned in the [previous section](#coreHTTP-Config-File), either a custom
config file (i.e. `core_http_config.h`) OR `HTTP_DO_NOT_USE_CUSTOM_CONFIG` macro
needs to be provided to build the HTTP client library.
For a CMake example of building the HTTP library with the `httpFilePaths.cmake`
file, refer to the `coverity_analysis` library target in
[test/CMakeLists.txt](test/CMakeLists.txt) file.
## Building Unit Tests
### Platform Prerequisites
- For running unit tests, the following are required:
- **C90 compiler** like gcc
- **CMake 3.13.0 or later**
- **Ruby 2.0.0 or later** is required for this repository's
[CMock test framework](https://github.com/ThrowTheSwitch/CMock).
- For running the coverage target, the following are required:
- **gcov**
- **lcov**
### Steps to build **Unit Tests**
1. Go to the root directory of this repository.
1. Run the *cmake* command: `cmake -S test -B build -DBUILD_CLONE_SUBMODULES=ON `
1. Run this command to build the library and unit tests: `make -C build all`
1. The generated test executables will be present in `build/bin/tests` folder.
1. Run `cd build && ctest` to execute all tests and view the test run summary.
## Reference examples
The AWS IoT Device SDK for Embedded C repository contains demos of using the HTTP client
library [here](https://github.com/aws/aws-iot-device-sdk-embedded-C/tree/main/demos/http)
on a POSIX platform. These can be used as reference examples for the library API.
## Documentation
### Existing Documentation
For pre-generated documentation, please see the documentation linked in the locations below:
| Location |
| :-: |
| [AWS IoT Device SDK for Embedded C](https://github.com/aws/aws-iot-device-sdk-embedded-C#releases-and-documentation) |
| [FreeRTOS.org](https://freertos.org/Documentation/api-ref/coreHTTP/docs/doxygen/output/html/index.html) |
Note that the latest included version of coreHTTP may differ across repositories.
### Generating Documentation
The Doxygen references were created using Doxygen version 1.9.2. To generate the
Doxygen pages, please run the following command from the root of this repository:
```shell
doxygen docs/doxygen/config.doxyfile
```
## Contributing
See [CONTRIBUTING.md](./.github/CONTRIBUTING.md) for information on contributing.

5
arcs-sdk/modules/coreHTTP/SECURITY.md Normal file
View File

@@ -0,0 +1,5 @@
## Reporting a Vulnerability
If you discover a potential security issue in this project, we ask that you notify AWS/Amazon Security
via our [vulnerability reporting page](https://aws.amazon.com/security/vulnerability-reporting/) or directly via email to aws-security@amazon.com.
Please do **not** create a public github issue.

2643
arcs-sdk/modules/coreHTTP/docs/doxygen/config.doxyfile Normal file
View File

File diff suppressed because it is too large Load Diff

BIN
arcs-sdk/modules/coreHTTP/docs/doxygen/images/httpclient_send_activity_diagram.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 208 KiB

35
arcs-sdk/modules/coreHTTP/docs/doxygen/images/httpclient_send_activity_diagram.pu Normal file
View File

@@ -0,0 +1,35 @@
@startuml
skinparam dpi 300
skinparam ArrowFontSize 18
start
#lightblue: **Send request on the Network**;
#lightgreen:if( Network error ) then (yes)
#lightgray: Return ERROR;
stop
else (no)
repeat
#lightblue: **Read response on the Network**;
#lightgreen:if( Network error ) then (yes)
#lightgray: Return ERROR;
stop
else (no)
endif
: Parse HTTP response;
note left: Parsing is done with\nthird-party //http-parser//
repeat while (Response buffer is NOT full &&
Response message is NOT complete &&
No errors found in parsing) is ( yes)
-> no;
#lightgreen:if (Errors found in parsing) then (yes)
#lightgray: Return ERROR;
stop
#lightgreen:(no ) elseif (Response message could not fit in response buffer) then (yes)
#lightgray: Return ERROR;
stop
else (no)
endif
: Return response;
stop
@enduml

BIN
arcs-sdk/modules/coreHTTP/docs/doxygen/images/httpclient_send_sequence_diagram.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 240 KiB

44
arcs-sdk/modules/coreHTTP/docs/doxygen/images/httpclient_send_sequence_diagram.pu Normal file
View File

@@ -0,0 +1,44 @@
@startuml
skinparam dpi 300
Application -> HTTP_Client_Lib: HTTPClient_Send(<font color=red>Request Headers</font>, <font color=blue>Request Body</font>, <font color=green>Response</font>)
activate HTTP_Client_Lib #lightgreen
HTTP_Client_Lib -> Application: transport send(<font color=red>Request Headers</font>)
activate Application #lightblue
Application -> Application: Send <font color=red>Request Headers</font> over the network.
Application --> HTTP_Client_Lib: Return SUCCESS
deactivate Application
note right: **transport send** is repeated if\nfewer bytes than requested to\nsend are sent.\nFor the sake of simplicity this\nexample returns successfully\nfrom the transport interface.
HTTP_Client_Lib -> Application: transport send(<font color=blue>Request Body</font>)
activate Application #lightblue
Application -> Application: Send <font color=blue>Request Body</font> over the network.
Application --> HTTP_Client_Lib: Return SUCCESS
deactivate Application
loop While <font color=darkgreen>response buffer</font> is NOT full &&\nResponse message is NOT complete &&\nNo errors found in parsing
HTTP_Client_Lib -> Application: transport read(<font color=darkgreen>Response buffer</font>)
activate Application #lightblue
Application -> Application: Receive network data into the <font color=darkgreen>Response buffer</font>.
Application --> HTTP_Client_Lib: Return SUCCESS
deactivate Application
HTTP_Client_Lib -> HTTP_Client_Lib: Parse <font color=green>Response</font>
activate HTTP_Client_Lib #darkgreen
loop for each response header found in parsing
HTTP_Client_Lib -> Application: __**onHeaderCallback**__(<font color=green>Response Header Field</font>,\n <font color=green>Response Header Value</font>,\n <font color=green>Response Status</font>)
activate Application #lightblue
Application -> Application: Do stuff with the Header
Application --> HTTP_Client_Lib: Return
deactivate Application
end
deactivate HTTP_Client_Lib
end
HTTP_Client_Lib --> Application: Return HTTP_RETURN_CODE
deactivate HTTP_Client_Lib
@enduml

25
arcs-sdk/modules/coreHTTP/docs/doxygen/include/size_table.md Normal file
View File

@@ -0,0 +1,25 @@
<table>
<tr>
<td colspan="3"><center><b>Code Size of coreHTTP (example generated with GCC for ARM Cortex-M)</b></center></td>
</tr>
<tr>
<td><b>File</b></td>
<td><b><center>With -O1 Optimization</center></b></td>
<td><b><center>With -Os Optimization</center></b></td>
</tr>
<tr>
<td>core_http_client.c</td>
<td><center>3.2K</center></td>
<td><center>2.6K</center></td>
</tr>
<tr>
<td>http_parser.c (http-parser)</td>
<td><center>15.7K</center></td>
<td><center>13.0K</center></td>
</tr>
<tr>
<td><b>Total estimates</b></td>
<td><b><center>18.9K</center></b></td>
<td><b><center>15.6K</center></b></td>
</tr>
</table>

228
arcs-sdk/modules/coreHTTP/docs/doxygen/layout.xml Normal file
View File

@@ -0,0 +1,228 @@
<doxygenlayout version="1.0">
<!-- Generated by doxygen 1.8.20 -->
<!-- Navigation index tabs for HTML output -->
<navindex>
<tab type="mainpage" visible="yes" title=""/>
<tab type="pages" visible="yes" title="" intro=""/>
<!-- Hide the default "Data Structures" tab and use the "Modules" tab for data
structures. This allows internal data structures to be hidden. -->
<tab type="modules" visible="yes" title="Data types and Constants" intro="This library defines the following data types and constants."/>
<tab type="namespaces" visible="yes" title="">
<tab type="namespacelist" visible="yes" title="" intro=""/>
<tab type="namespacemembers" visible="yes" title="" intro=""/>
</tab>
<tab type="interfaces" visible="no" title="">
<tab type="interfacelist" visible="no" title="" intro=""/>
<tab type="interfaceindex" visible="no" title=""/>
<tab type="interfacehierarchy" visible="yes" title="" intro=""/>
</tab>
<tab type="classes" visible="no" title="">
<tab type="classlist" visible="no" title="" intro=""/>
<tab type="classindex" visible="no" title=""/>
<tab type="hierarchy" visible="no" title="" intro=""/>
<tab type="classmembers" visible="no" title="" intro=""/>
</tab>
<tab type="structs" visible="no" title="">
<tab type="structlist" visible="no" title="" intro=""/>
<tab type="structindex" visible="no" title=""/>
</tab>
<tab type="exceptions" visible="no" title="">
<tab type="exceptionlist" visible="no" title="" intro=""/>
<tab type="exceptionindex" visible="no" title=""/>
<tab type="exceptionhierarchy" visible="yes" title="" intro=""/>
</tab>
<tab type="files" visible="no" title="">
<tab type="filelist" visible="yes" title="Files" intro="The following files are associated with this library."/>
<tab type="globals" visible="no" title="" intro=""/>
</tab>
<tab type="examples" visible="yes" title="" intro=""/>
</navindex>
<!-- Layout definition for a class page -->
<class>
<briefdescription visible="yes"/>
<includes visible="$SHOW_INCLUDE_FILES"/>
<inheritancegraph visible="$CLASS_GRAPH"/>
<collaborationgraph visible="$COLLABORATION_GRAPH"/>
<memberdecl>
<nestedclasses visible="yes" title=""/>
<publictypes title=""/>
<services title=""/>
<interfaces title=""/>
<publicslots title=""/>
<signals title=""/>
<publicmethods title=""/>
<publicstaticmethods title=""/>
<publicattributes title=""/>
<publicstaticattributes title=""/>
<protectedtypes title=""/>
<protectedslots title=""/>
<protectedmethods title=""/>
<protectedstaticmethods title=""/>
<protectedattributes title=""/>
<protectedstaticattributes title=""/>
<packagetypes title=""/>
<packagemethods title=""/>
<packagestaticmethods title=""/>
<packageattributes title=""/>
<packagestaticattributes title=""/>
<properties title=""/>
<events title=""/>
<privatetypes title=""/>
<privateslots title=""/>
<privatemethods title=""/>
<privatestaticmethods title=""/>
<privateattributes title=""/>
<privatestaticattributes title=""/>
<friends title=""/>
<related title="" subtitle=""/>
<membergroups visible="yes"/>
</memberdecl>
<detaileddescription title=""/>
<memberdef>
<inlineclasses title=""/>
<typedefs title=""/>
<enums title=""/>
<services title=""/>
<interfaces title=""/>
<constructors title=""/>
<functions title=""/>
<related title=""/>
<variables title=""/>
<properties title=""/>
<events title=""/>
</memberdef>
<allmemberslink visible="yes"/>
<usedfiles visible="$SHOW_USED_FILES"/>
<authorsection visible="yes"/>
</class>
<!-- Layout definition for a namespace page -->
<namespace>
<briefdescription visible="yes"/>
<memberdecl>
<nestednamespaces visible="yes" title=""/>
<constantgroups visible="yes" title=""/>
<interfaces visible="yes" title=""/>
<classes visible="yes" title=""/>
<structs visible="yes" title=""/>
<exceptions visible="yes" title=""/>
<typedefs title=""/>
<sequences title=""/>
<dictionaries title=""/>
<enums title=""/>
<functions title=""/>
<variables title=""/>
<membergroups visible="yes"/>
</memberdecl>
<detaileddescription title=""/>
<memberdef>
<inlineclasses title=""/>
<typedefs title=""/>
<sequences title=""/>
<dictionaries title=""/>
<enums title=""/>
<functions title=""/>
<variables title=""/>
</memberdef>
<authorsection visible="yes"/>
</namespace>
<!-- Layout definition for a file page -->
<file>
<briefdescription visible="yes"/>
<includes visible="$SHOW_INCLUDE_FILES"/>
<includegraph visible="$INCLUDE_GRAPH"/>
<includedbygraph visible="$INCLUDED_BY_GRAPH"/>
<sourcelink visible="yes"/>
<memberdecl>
<interfaces visible="yes" title=""/>
<classes visible="yes" title=""/>
<structs visible="yes" title=""/>
<exceptions visible="yes" title=""/>
<namespaces visible="yes" title=""/>
<constantgroups visible="yes" title=""/>
<defines title=""/>
<typedefs title=""/>
<sequences title=""/>
<dictionaries title=""/>
<enums title=""/>
<functions title=""/>
<variables title=""/>
<membergroups visible="yes"/>
</memberdecl>
<detaileddescription title=""/>
<memberdef>
<inlineclasses title=""/>
<defines title=""/>
<typedefs title=""/>
<sequences title=""/>
<dictionaries title=""/>
<enums title=""/>
<functions title=""/>
<variables title=""/>
</memberdef>
<authorsection/>
</file>
<!-- Layout definition for a group page -->
<group>
<briefdescription visible="yes"/>
<groupgraph visible="$GROUP_GRAPHS"/>
<memberdecl>
<nestedgroups visible="yes" title=""/>
<dirs visible="yes" title=""/>
<files visible="yes" title=""/>
<namespaces visible="yes" title=""/>
<classes visible="yes" title=""/>
<defines title=""/>
<typedefs title=""/>
<sequences title=""/>
<dictionaries title=""/>
<enums title=""/>
<enumvalues title=""/>
<functions title=""/>
<variables title=""/>
<signals title=""/>
<publicslots title=""/>
<protectedslots title=""/>
<privateslots title=""/>
<events title=""/>
<properties title=""/>
<friends title=""/>
<membergroups visible="yes"/>
</memberdecl>
<detaileddescription title=""/>
<memberdef>
<pagedocs/>
<inlineclasses title=""/>
<defines title=""/>
<typedefs title=""/>
<sequences title=""/>
<dictionaries title=""/>
<enums title=""/>
<enumvalues title=""/>
<functions title=""/>
<variables title=""/>
<signals title=""/>
<publicslots title=""/>
<protectedslots title=""/>
<privateslots title=""/>
<events title=""/>
<properties title=""/>
<friends title=""/>
</memberdef>
<authorsection visible="yes"/>
</group>
<!-- Layout definition for a directory page -->
<directory>
<briefdescription visible="yes"/>
<directorygraph visible="yes"/>
<memberdecl>
<dirs visible="yes"/>
<files visible="yes"/>
</memberdecl>
<detaileddescription title=""/>
</directory>
</doxygenlayout>

332
arcs-sdk/modules/coreHTTP/docs/doxygen/pages.dox Normal file
View File

@@ -0,0 +1,332 @@
/**
@mainpage Overview
@anchor http
@brief HTTP Client library
This HTTP Client library implements a subset of the HTTP/1.1 protocol. Features
of this library include:
- Fully synchronous API, to allow applications to completely manage their concurrency and multi-threading.
- Operations on user supplied buffers, so that applications have complete control of their memory allocation strategy.
- Integration with [http-parser](https://github.com/nodejs/http-parser) to handle chunked encoding.
Feature of HTTP/1.1 not supported in this library:
- Streaming uploads and downloads. Range requests for partial content responses are highly encouraged with this API.
- Pipelining requests. There may be only one request outgoing and one response incoming, at a time, on a connection.
- Automatic redirection. The user application owns their connection and must handle redirection status codes.
@section http_memory_requirements Memory Requirements
@brief Memory requirements of the HTTP Client library.
@include{doc} size_table.md
*/
/**
@page http_design Design
HTTP Client Library Architecture and Design
This HTTP client library implements a subset of the HTTP/1.1 protocol. It is
optimized for resource constrained devices and does not dynamically allocate any
memory.
@section http_transport_interface_blurb Transport Interface
For decoupling from the user platform, the HTTP client library uses a
transport interface. The transport interface allows the HTTP client library to
send and receive data over the user's transport layer. The user platform must
implement a @ref TransportInterface_t to use in @ref HTTPClient_Send.
@see The [transport interface documentation](@ref http_transport_interface) for
more information.
@section http_request_serialization Building an HTTP Request
The HTTP client library provides the following API to serialize request headers.
- @ref HTTPClient_InitializeRequestHeaders
- @ref HTTPClient_AddHeader
- @ref HTTPClient_AddRangeHeader
An application is expected to create and populate an @ref HTTPRequestInfo_t and
set a buffer to use for the headers in @ref HTTPRequestHeaders_t.pBuffer. The
application may first call @ref HTTPClient_InitializeRequestHeaders to populate
the @ref HTTPRequestHeaders_t with the method, the path, and the host. The
HTTP request will be serialized to the following when HTTPRequestInfo_t.reqFlags
is zero.
@code
<METHOD> <PATH> HTTP/1.1\r\n
User-Agent: <MY-PLATFORM>\r\n
Host: <SERVER-URL>\r\n\r\n
@endcode
When the @ref HTTPRequestInfo_t.reqFlags has @ref HTTP_REQUEST_KEEP_ALIVE_FLAG
set, then the HTTP request will be serialized to the following:
@code
<METHOD> <PATH> HTTP/1.1\r\n
User-Agent: <MY-PLATFORM>\r\n
Host: <SERVER-URL>\r\n
Connection: keep-alive\r\n\r\n
@endcode
The user application may add more headers using @ref HTTPClient_AddHeader or
@ref HTTPClient_AddRangeHeader. New headers will be appended to the end of the
existing headers. Please see the following example:
@code
<METHOD> <PATH> HTTP/1.1\r\n
User-Agent: <MY-PLATFORM>\r\n
Host: <SERVER-URL>\r\n
Connection: keep-alive\r\n
Another-Header1: another-value1\r\n
Another-Header2: another-value2\r\n
Another-Header3: another-value3\r\n\r\n
@endcode
The user application may pass a request body into the @ref HTTPClient_Send
function when the request is ready to be sent.
@section http_range_support HTTP Range Requests and Partial Content Responses
Range Requests are strongly encouraged for downloading a large file. Large is
defined here to be a file whose total size cannot fit into the space currently
available in RAM. By downloading a large file using range requests the user
application can spend time processing that part of the file (for example writing
to flash), then request the next part of the file. If the user application were
to request the entire file at once and process it in sections from the network,
the system is at a high risk for dropping packets. Dropped packets cause
retransmissions in the system's transport layer. With many, there can be a
negative impact on the overall system throughput, network bandwidth, and battery
life of the device.
Range requests are supported using @ref HTTPClient_AddRangeHeader. Please see
the function documentation for more information.
@section http_response_deserialization Receiving and Parsing an HTTP Response
After the request headers are serialized, the user application must set a buffer
to receive the HTTP response in @ref HTTPResponse_t.pBuffer.
@ref HTTPClient_Send is then used to send the request and receive the response.
If the request has a body it is passed as a parameter to @ref HTTPClient_Send.
As soon as the response is received from the network it is parsed. The final
parsed response is represented by the @ref HTTPResponse_t returned from
@ref HTTPClient_Send. Parsing the HTTP response is done using
[http-parser](https://github.com/nodejs/http-parser). http-parser invokes
callbacks for each section in the HTTP response it finds. Using these callbacks
the HTTP client library sets the members of @ref HTTPResponse_t to return from
@ref HTTPClient_Send. The overall flow of @ref HTTPClient_Send is
shown in the activity diagram below:
@image html httpclient_send_activity_diagram.png width=50%
@section http_response_headers Reading the HTTP Response Headers
Upon a successful return from @ref HTTPClient_Send, the HTTP Response headers
can be read from the headers found in @ref HTTPResponse_t.pHeaders. The function
@ref HTTPClient_ReadHeader reads the headers from an @ref HTTPResponse_t.
@ref HTTPClient_ReadHeader will re-parse the response in
@ref HTTPResponse_t.pBuffer, looking for the header field of interest.
Re-parsing involves using http-parser to look at each character starting from
the beginning of @ref HTTPResponse_t.pBuffer until the header field of interest
is found.
If the user application wants to avoid re-parsing @ref HTTPResponse_t.pBuffer,
then the user application may register a callback in
@ref HTTPResponse_t.pHeaderParsingCallback. When the HTTP response message is
first received from the network, in @ref HTTPClient_Send, http-parser is invoked
to parse the response. This first parsing in @ref HTTPClient_Send will invoke
@ref HTTPResponse_t.pHeaderParsingCallback for each header that is found
in response. Please see the sequence diagram below for an illustration of when
@ref HTTPClient_ResponseHeaderParsingCallback_t.onHeaderCallback is invoked
during the operation of @ref HTTPClient_Send.
@image html httpclient_send_sequence_diagram.png width=50%
*/
/**
@page http_porting Porting Guide
@brief Guide for porting the HTTP client library to a new platform.
To use the HTTP client library, a platform must implement the following
components:
1. [Configuration Macros](@ref http_porting_config)
2. [Transport Interface](@ref http_porting_transport)
@section http_porting_config Configuration Macros
@brief Settings that can be set as macros in the config header
`core_http_config.h`, or passed in as compiler options.
@note If the custom configuration header `core_http_config.h` is not provided,
then the @ref HTTP_DO_NOT_USE_CUSTOM_CONFIG macro must be defined.
@see [Configurations](@ref http_config)
The following macros can be configured for this library:
- @ref HTTP_MAX_RESPONSE_HEADERS_SIZE_BYTES
- @ref HTTP_USER_AGENT_VALUE
- @ref HTTP_SEND_RETRY_TIMEOUT_MS
- @ref HTTP_RECV_RETRY_TIMEOUT_MS
In addition, the following logging macros are used throughout this library:
- @ref LogError
- @ref LogWarn
- @ref LogInfo
- @ref LogDebug
@section http_porting_transport Transport Interface
@brief The HTTP client library relies on transport interface callbacks
that must be implemented in order to send and receive packets on a network.
@see The [Transport Interface](@ref http_transport_interface) documentation for
more information.
The transport interface API used by the HTTP client is defined in
@ref transport_interface.h. A port must implement functions corresponding to the
following functions pointers:
- [Transport Receive](@ref TransportRecv_t): A function to receive bytes from a network.
@code
int32_t (* TransportRecv_t )(
NetworkContext_t * pNetworkContext, void * pBuffer, size_t bytesToRecv
);
@endcode
- [Transport Send](@ref TransportSend_t): A function to send bytes over a network.
@code
int32_t (* TransportSend_t )(
NetworkContext_t * pNetworkContext, const void * pBuffer, size_t bytesToSend
);
@endcode
The above two functions take in a pointer to a @ref NetworkContext_t, the type
name of a `struct NetworkContext`. The NetworkContext struct must also be
defined by the user's implementation, and ought to contain any information
necessary to send and receive data with the @ref TransportSend_t and
@ref TransportRecv_t implementations, respectively:
@code
struct NetworkContext {
// Fields necessary for the transport implementations, e.g. a TCP socket descriptor.
};
@endcode
@section http_porting_time Time Function
@brief The HTTP library optionally relies on a function to generate millisecond
timestamps, for the purpose of calculating the elapsed time when no data has
been sent or received.
@see @ref HTTPClient_GetCurrentTimeFunc_t
Applications can supply their platform-specific function capable of generating
32-bit timestamps of millisecond resolution. These timestamps need not correspond
with any real world clock; the only requirement is that the difference between two
timestamps must be an accurate representation of the duration between them, in
milliseconds.
This function is used in conjunction with macros @ref HTTP_SEND_RETRY_TIMEOUT_MS
and @ref HTTP_RECV_RETRY_TIMEOUT_MS.
*/
/**
@page http_config Configurations
@brief Configurations of the HTTP Client library.
<!-- @par configpagestyle allows the @section titles to be styled according to style.css -->
@par configpagestyle
Configuration settings are C pre-processor constants. They can be set with a \#define in the config file (core_http_config.h) or by using a compiler option such as -D in gcc.
@section HTTP_DO_NOT_USE_CUSTOM_CONFIG
@brief Define this macro to build the HTTP client library without the custom
config file core_http_config.h.
Without the custom config, the HTTP client library builds with default values of
config macros defined in core_http_config_defaults.h file.
If a custom config is provided, then HTTP_DO_NOT_USE_CUSTOM_CONFIG should not be
defined.
@section HTTP_MAX_RESPONSE_HEADERS_SIZE_BYTES
@copydoc HTTP_MAX_RESPONSE_HEADERS_SIZE_BYTES
@section HTTP_USER_AGENT_VALUE
@copydoc HTTP_USER_AGENT_VALUE
@section HTTP_SEND_RETRY_TIMEOUT_MS
@copydoc HTTP_SEND_RETRY_TIMEOUT_MS
@section HTTP_RECV_RETRY_TIMEOUT_MS
@copydoc HTTP_RECV_RETRY_TIMEOUT_MS
@section http_logerror LogError
@copydoc LogError
@section http_logwarn LogWarn
@copydoc LogWarn
@section http_loginfo LogInfo
@copydoc LogInfo
@section http_logdebug LogDebug
@copydoc LogDebug
*/
/**
@page http_functions Functions
@brief Primary functions of the HTTP Client library:<br><br>
@subpage httpclient_initializerequestheaders_function <br>
@subpage httpclient_addheader_function <br>
@subpage httpclient_addrangeheader_function <br>
@subpage httpclient_send_function <br>
@subpage httpclient_readheader_function <br>
@subpage httpclient_strerror_function <br>
@page httpclient_initializerequestheaders_function HTTPClient_InitializeRequestHeaders
@snippet core_http_client.h declare_httpclient_initializerequestheaders
@copydoc HTTPClient_InitializeRequestHeaders
@page httpclient_addheader_function HTTPClient_AddHeader
@snippet core_http_client.h declare_httpclient_addheader
@copydoc HTTPClient_AddHeader
@page httpclient_addrangeheader_function HTTPClient_AddRangeHeader
@snippet core_http_client.h declare_httpclient_addrangeheader
@copydoc HTTPClient_AddRangeHeader
@page httpclient_send_function HTTPClient_Send
@snippet core_http_client.h declare_httpclient_send
@copydoc HTTPClient_Send
@page httpclient_readheader_function HTTPClient_ReadHeader
@snippet core_http_client.h declare_httpclient_readheader
@copydoc HTTPClient_ReadHeader
@page httpclient_strerror_function HTTPClient_strerror
@snippet core_http_client.h declare_httpclient_strerror
@copydoc HTTPClient_strerror
*/
<!-- We do not use doxygen ALIASes here because there have been issues in the past versions with "^^" newlines within the alias definition. -->
/**
@defgroup http_enum_types Enumerated Types
@brief Enumerated types of the HTTP Client library
*/
/**
@defgroup http_callback_types Callback Types
@brief Callback function pointer types of the HTTP Client library
*/
/**
@defgroup http_struct_types Parameter Structures
@brief Structures passed as parameters to [HTTP Client library functions](@ref http_functions)
These structures are passed as parameters to library functions. Documentation for these structures will state the functions associated with each parameter structure and the purpose of each member.
*/
/**
@defgroup http_basic_types Basic Types
@brief Primitive types of the HTTP Client library.
*/
/**
@defgroup http_constants Constants
@brief Constants defined in the HTTP Client library
*/

132
arcs-sdk/modules/coreHTTP/docs/doxygen/style.css Normal file
View File

@@ -0,0 +1,132 @@
/*
* Stylesheet for Doxygen HTML output.
*
* This file defines styles for custom elements in the header/footer and
* overrides some of the default Doxygen styles.
*
* Styles in this file do not affect the treeview sidebar.
*/
/* Set the margins to place a small amount of whitespace on the left and right
* side of the page. */
div.contents {
margin-left:4em;
margin-right:4em;
}
/* Justify text in paragraphs. */
p {
text-align: justify;
}
/* Style of section headings. */
h1 {
border-bottom: 1px solid #879ECB;
color: #354C7B;
font-size: 160%;
font-weight: normal;
padding-bottom: 4px;
padding-top: 8px;
}
/* Style of subsection headings. */
h2:not(.memtitle):not(.groupheader) {
font-size: 125%;
margin-bottom: 0px;
margin-top: 16px;
padding: 0px;
}
/* Style of paragraphs immediately after subsection headings. */
h2 + p {
margin: 0px;
padding: 0px;
}
/* Style of subsection headings. */
h3 {
font-size: 100%;
margin-bottom: 0px;
margin-left: 2em;
margin-right: 2em;
}
/* Style of paragraphs immediately after subsubsection headings. */
h3 + p {
margin-top: 0px;
margin-left: 2em;
margin-right: 2em;
}
/* Style of the prefix "AWS IoT Device SDK C" that appears in the header. */
#csdkprefix {
color: #757575;
}
/* Style of the "Return to main page" link that appears in the header. */
#returntomain {
padding: 0.5em;
}
/* Style of the dividers on Configuration Settings pages. */
div.configpagedivider {
margin-left: 0px !important;
margin-right: 0px !important;
margin-top: 20px !important;
}
/* Style of configuration setting names. */
dl.section.user ~ h1 {
border-bottom: none;
color: #000000;
font-family: monospace, fixed;
font-size: 16px;
margin-bottom: 0px;
margin-left: 2em;
margin-top: 1.5em;
}
/* Style of paragraphs on a configuration settings page. */
dl.section.user ~ * {
margin-bottom: 10px;
margin-left: 4em;
margin-right: 4em;
margin-top: 0px;
}
/* Hide the configuration setting marker. */
dl.section.user {
display: none;
}
/* Overrides for code fragments and lines. */
div.fragment {
background: #ffffff;
border: none;
padding: 5px;
}
div.line {
color: #3a3a3a;
}
/* Overrides for code syntax highlighting colors. */
span.comment {
color: #008000;
}
span.keyword, span.keywordtype, span.keywordflow {
color: #0000ff;
}
span.preprocessor {
color: #50015a;
}
span.stringliteral, span.charliteral {
color: #800c0c;
}
a.code, a.code:visited, a.line, a.line:visited {
color: #496194;
}

17
arcs-sdk/modules/coreHTTP/httpFilePaths.cmake Normal file
View File

@@ -0,0 +1,17 @@
# This file is to add source files and include directories
# into variables so that it can be reused from different repositories
# in their Cmake based build system by including this file.
#
# Files specific to the repository such as test runner, platform tests
# are not added to the variables.
# HTTP library source files.
set( HTTP_SOURCES
${CMAKE_CURRENT_LIST_DIR}/source/core_http_client.c
${CMAKE_CURRENT_LIST_DIR}/source/dependency/3rdparty/http_parser/http_parser.c )
# HTTP library public include directories.
set( HTTP_INCLUDE_PUBLIC_DIRS
${CMAKE_CURRENT_LIST_DIR}/source/include
${CMAKE_CURRENT_LIST_DIR}/source/interface
${CMAKE_CURRENT_LIST_DIR}/source/dependency/3rdparty/http_parser )

275
arcs-sdk/modules/coreHTTP/lexicon.txt Normal file
View File

@@ -0,0 +1,275 @@
absolutevalue
addheader
addrangeheader
addrangeheaders
addrangerequest
addtogroup
aggregator
api
apis
ascii
aws
bool
br
bufferlen
bufferlength
bytesreceived
bytesremaining
bytessent
bytestorecv
bytestosend
calltlsrecvfunc
cb
cbmc
chk
chunked
colspan
com
config
configpagestyle
const
contentlength
convertint
copybrief
copydoc
corehttp
coverity
cprover
css
datalen
datelen
defgroup
doesn
doxygen
endcode
endif
enums
eof
errno
expectedheader
fieldfound
fieldlen
fieldlentoreturn
fieldloc
findheadercontext
findheaderfieldparsercallback
findheaderinresponse
findheaderonheadercompletecallback
findheadervalueparsercallback
firstpartbytes
gcc
getfinalresponsestatus
gettime
gettimestampms
github
headercount
headerslen
hostlen
hpe
html
http
httpclient
httpheadernotfound
httpheaderstrncpy
httpinsufficientmemory
httpinvalidparameter
httpinvalidresponse
httplibrarystatus
httpnetworkerror
httpnoresponse
httpparseonstatusfieldcallback
httpparser
httpparserinternalerror
httpparseronbodycallback
httpparseronheaderfieldcallback
httpparseronheaderscompletecallback
httpparseronheadervaluecallback
httpparseronmessagebegincallback
httpparseronmessagecompletecallback
httpparseronstatuscallback
httpparserxxxxcallback
httpparserxxxxcallbacks
httpparsingcontext
httpparsingstate
httppartialresponse
httprequestheaders
httprequestinfo
httpresponse
https
httpsecurityalertextraneousresponsedata
httpsecurityalertinvalidcontentlength
httpsecurityalertinvalidcharacter
httpsecurityalertinvalidchunkheader
httpsecurityalertinvalidprotocolversion
httpsecurityalertinvalidstatuscode
httpsecurityalertresponseheaderssizelimitexceeded
httpstatus
httpsuccess
ietf
ifndef
inc
ingroup
init
initializerequestheaders
int
iot
isfield
isheaderresponse
isheadresponse
iso
lastheaderfieldlen
lastheadervaluelen
latin
len
linux
logdebug
logerror
loginfo
logwarn
mainpage
malloc
md
memcpy
memmove
methodlen
min
misra
mqtt
msg
mynetworkrecvimplementation
mynetworksendimplementation
myplatformnetworkcontext
myplatformtransportreceive
myplatformtransportsend
mytcpsocketcontext
mytlscontext
networkcontext
nodejs
noninfringement
ok
onbody
onheadercallback
onheaderfield
onheaderscomplete
onheadervalue
onmessagebegin
onmessagecomplete
onstatus
ored
org
os
param
parsehttpresponse
parselen
parsersettings
parsingstate
pathlen
pbuffer
pbuffercur
pbytesreceived
pcontext
pdata
pdateloc
pdest
pfield
pfieldloc
pfindheadercontext
pheaderparsingcallback
pheaders
phost
phttpparser
phttpparsingcontext
plaintext
plastheaderfield
plastheadervalue
ploc
psrc
pmethod
pname
pnetworkcontext
pnetworkdata
pnext
pnextwriteloc
png
posix
pparsingcontext
pparsingstate
ppath
pre
prefill
prequestbodybuf
prequestheaders
prequestinfo
presponse
processhttpparsererror
ptransport
ptransportinterface
pvalue
pvaluelen
pvalueloc
rangeend
rangestart
rangestartorlastnbytes
readheader
receivehttpdata
recv
recvcurrentcall
recvstopcall
recvtimeoutcall
reponse
reqbodybuflen
reqbodylen
reqflags
requestbody
requestheaderbuffer
requestheaders
requestinfo
respflags
responsebufferlen
rfc
rm
sdk
senderrorcall
sendflags
sendhttpbody
sendhttpheaders
sendpartialcall
sensitivity
sizeof
snprintf
statuscode
strchr
strerror
strncpy
struct
sublicense
tcp
tcpsocketcontext
td
tls
tlscontext
tlsrecv
tlsrecvcount
tlssend
toascii
toolchain
totalreceived
tr
transportcallback
transportinterface
transportpage
transportrecv
transportsectionimplementation
transportsectionoverview
transportsend
transportstatus
transportstruct
tx
txt
uint
uri
url
valuefound
valuelen
valueloc
xxxx

11
arcs-sdk/modules/coreHTTP/manifest.yml Normal file
View File

@@ -0,0 +1,11 @@
name : "coreHTTP"
version: "v2.1.0"
description: |
"Client implementation of the HTTP/1.1 specification for embedded devices.\n"
dependencies:
- name : "http-parser"
version: "v2.9.4"
repository:
type: "git"
url: "https://github.com/nodejs/http-parser"
license: "MIT"

2912
arcs-sdk/modules/coreHTTP/source/core_http_client.c Normal file
View File

File diff suppressed because it is too large Load Diff

8
arcs-sdk/modules/coreHTTP/source/dependency/3rdparty/http_parser/.mailmap vendored Normal file
View File

@@ -0,0 +1,8 @@
# update AUTHORS with:
# git log --all --reverse --format='%aN <%aE>' | perl -ne 'BEGIN{print "# Authors ordered by first contribution.\n"} print unless $h{$_}; $h{$_} = 1' > AUTHORS
Ryan Dahl <ry@tinyclouds.org>
Salman Haq <salman.haq@asti-usa.com>
Simon Zimmermann <simonz05@gmail.com>
Thomas LE ROUX <thomas@november-eleven.fr> LE ROUX Thomas <thomas@procheo.fr>
Thomas LE ROUX <thomas@november-eleven.fr> Thomas LE ROUX <thomas@procheo.fr>
Fedor Indutny <fedor@indutny.com>

13
arcs-sdk/modules/coreHTTP/source/dependency/3rdparty/http_parser/.travis.yml vendored Normal file
View File

@@ -0,0 +1,13 @@
language: c
compiler:
- clang
- gcc
script:
- "make"
notifications:
email: false
irc:
- "irc.freenode.net#node-ci"

68
arcs-sdk/modules/coreHTTP/source/dependency/3rdparty/http_parser/AUTHORS vendored Normal file
View File

@@ -0,0 +1,68 @@
# Authors ordered by first contribution.
Ryan Dahl <ry@tinyclouds.org>
Jeremy Hinegardner <jeremy@hinegardner.org>
Sergey Shepelev <temotor@gmail.com>
Joe Damato <ice799@gmail.com>
tomika <tomika_nospam@freemail.hu>
Phoenix Sol <phoenix@burninglabs.com>
Cliff Frey <cliff@meraki.com>
Ewen Cheslack-Postava <ewencp@cs.stanford.edu>
Santiago Gala <sgala@apache.org>
Tim Becker <tim.becker@syngenio.de>
Jeff Terrace <jterrace@gmail.com>
Ben Noordhuis <info@bnoordhuis.nl>
Nathan Rajlich <nathan@tootallnate.net>
Mark Nottingham <mnot@mnot.net>
Aman Gupta <aman@tmm1.net>
Tim Becker <tim.becker@kuriositaet.de>
Sean Cunningham <sean.cunningham@mandiant.com>
Peter Griess <pg@std.in>
Salman Haq <salman.haq@asti-usa.com>
Cliff Frey <clifffrey@gmail.com>
Jon Kolb <jon@b0g.us>
Fouad Mardini <f.mardini@gmail.com>
Paul Querna <pquerna@apache.org>
Felix Geisendörfer <felix@debuggable.com>
koichik <koichik@improvement.jp>
Andre Caron <andre.l.caron@gmail.com>
Ivo Raisr <ivosh@ivosh.net>
James McLaughlin <jamie@lacewing-project.org>
David Gwynne <loki@animata.net>
Thomas LE ROUX <thomas@november-eleven.fr>
Randy Rizun <rrizun@ortivawireless.com>
Andre Louis Caron <andre.louis.caron@usherbrooke.ca>
Simon Zimmermann <simonz05@gmail.com>
Erik Dubbelboer <erik@dubbelboer.com>
Martell Malone <martellmalone@gmail.com>
Bertrand Paquet <bpaquet@octo.com>
BogDan Vatra <bogdan@kde.org>
Peter Faiman <peter@thepicard.org>
Corey Richardson <corey@octayn.net>
Tóth Tamás <tomika_nospam@freemail.hu>
Cam Swords <cam.swords@gmail.com>
Chris Dickinson <christopher.s.dickinson@gmail.com>
Uli Köhler <ukoehler@btronik.de>
Charlie Somerville <charlie@charliesomerville.com>
Patrik Stutz <patrik.stutz@gmail.com>
Fedor Indutny <fedor.indutny@gmail.com>
runner <runner.mei@gmail.com>
Alexis Campailla <alexis@janeasystems.com>
David Wragg <david@wragg.org>
Vinnie Falco <vinnie.falco@gmail.com>
Alex Butum <alexbutum@linux.com>
Rex Feng <rexfeng@gmail.com>
Alex Kocharin <alex@kocharin.ru>
Mark Koopman <markmontymark@yahoo.com>
Helge Heß <me@helgehess.eu>
Alexis La Goutte <alexis.lagoutte@gmail.com>
George Miroshnykov <george.miroshnykov@gmail.com>
Maciej Małecki <me@mmalecki.com>
Marc O'Morain <github.com@marcomorain.com>
Jeff Pinner <jpinner@twitter.com>
Timothy J Fontaine <tjfontaine@gmail.com>
Akagi201 <akagi201@gmail.com>
Romain Giraud <giraud.romain@gmail.com>
Jay Satiro <raysatiro@yahoo.com>
Arne Steen <Arne.Steen@gmx.de>
Kjell Schubert <kjell.schubert@gmail.com>
Olivier Mengué <dolmen@cpan.org>

19
arcs-sdk/modules/coreHTTP/source/dependency/3rdparty/http_parser/LICENSE-MIT vendored Normal file
View File

@@ -0,0 +1,19 @@
Copyright Joyent, Inc. and other Node contributors.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to
deal in the Software without restriction, including without limitation the
rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
sell copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
IN THE SOFTWARE.

160
arcs-sdk/modules/coreHTTP/source/dependency/3rdparty/http_parser/Makefile vendored Normal file
View File

@@ -0,0 +1,160 @@
# Copyright Joyent, Inc. and other Node contributors. All rights reserved.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to
# deal in the Software without restriction, including without limitation the
# rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
# sell copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
# IN THE SOFTWARE.
PLATFORM ?= $(shell sh -c 'uname -s | tr "[A-Z]" "[a-z]"')
HELPER ?=
BINEXT ?=
SOLIBNAME = libhttp_parser
SOMAJOR = 2
SOMINOR = 9
SOREV = 4
ifeq (darwin,$(PLATFORM))
SOEXT ?= dylib
SONAME ?= $(SOLIBNAME).$(SOMAJOR).$(SOMINOR).$(SOEXT)
LIBNAME ?= $(SOLIBNAME).$(SOMAJOR).$(SOMINOR).$(SOREV).$(SOEXT)
else ifeq (wine,$(PLATFORM))
CC = winegcc
BINEXT = .exe.so
HELPER = wine
else
SOEXT ?= so
SONAME ?= $(SOLIBNAME).$(SOEXT).$(SOMAJOR).$(SOMINOR)
LIBNAME ?= $(SOLIBNAME).$(SOEXT).$(SOMAJOR).$(SOMINOR).$(SOREV)
endif
CC?=gcc
AR?=ar
CPPFLAGS ?=
LDFLAGS ?=
CPPFLAGS += -I.
CPPFLAGS_DEBUG = $(CPPFLAGS) -DHTTP_PARSER_STRICT=1
CPPFLAGS_DEBUG += $(CPPFLAGS_DEBUG_EXTRA)
CPPFLAGS_FAST = $(CPPFLAGS) -DHTTP_PARSER_STRICT=0
CPPFLAGS_FAST += $(CPPFLAGS_FAST_EXTRA)
CPPFLAGS_BENCH = $(CPPFLAGS_FAST)
CFLAGS += -Wall -Wextra -Werror
CFLAGS_DEBUG = $(CFLAGS) -O0 -g $(CFLAGS_DEBUG_EXTRA)
CFLAGS_FAST = $(CFLAGS) -O3 $(CFLAGS_FAST_EXTRA)
CFLAGS_BENCH = $(CFLAGS_FAST) -Wno-unused-parameter
CFLAGS_LIB = $(CFLAGS_FAST) -fPIC
LDFLAGS_LIB = $(LDFLAGS) -shared
INSTALL ?= install
PREFIX ?= /usr/local
LIBDIR = $(PREFIX)/lib
INCLUDEDIR = $(PREFIX)/include
ifeq (darwin,$(PLATFORM))
LDFLAGS_LIB += -Wl,-install_name,$(LIBDIR)/$(SONAME)
else
# TODO(bnoordhuis) The native SunOS linker expects -h rather than -soname...
LDFLAGS_LIB += -Wl,-soname=$(SONAME)
endif
test: test_g test_fast
$(HELPER) ./test_g$(BINEXT)
$(HELPER) ./test_fast$(BINEXT)
test_g: http_parser_g.o test_g.o
$(CC) $(CFLAGS_DEBUG) $(LDFLAGS) http_parser_g.o test_g.o -o $@
test_g.o: test.c http_parser.h Makefile
$(CC) $(CPPFLAGS_DEBUG) $(CFLAGS_DEBUG) -c test.c -o $@
http_parser_g.o: http_parser.c http_parser.h Makefile
$(CC) $(CPPFLAGS_DEBUG) $(CFLAGS_DEBUG) -c http_parser.c -o $@
test_fast: http_parser.o test.o http_parser.h
$(CC) $(CFLAGS_FAST) $(LDFLAGS) http_parser.o test.o -o $@
test.o: test.c http_parser.h Makefile
$(CC) $(CPPFLAGS_FAST) $(CFLAGS_FAST) -c test.c -o $@
bench: http_parser.o bench.o
$(CC) $(CFLAGS_BENCH) $(LDFLAGS) http_parser.o bench.o -o $@
bench.o: bench.c http_parser.h Makefile
$(CC) $(CPPFLAGS_BENCH) $(CFLAGS_BENCH) -c bench.c -o $@
http_parser.o: http_parser.c http_parser.h Makefile
$(CC) $(CPPFLAGS_FAST) $(CFLAGS_FAST) -c http_parser.c
test-run-timed: test_fast
while(true) do time $(HELPER) ./test_fast$(BINEXT) > /dev/null; done
test-valgrind: test_g
valgrind ./test_g
libhttp_parser.o: http_parser.c http_parser.h Makefile
$(CC) $(CPPFLAGS_FAST) $(CFLAGS_LIB) -c http_parser.c -o libhttp_parser.o
library: libhttp_parser.o
$(CC) $(LDFLAGS_LIB) -o $(LIBNAME) $<
package: http_parser.o
$(AR) rcs libhttp_parser.a http_parser.o
url_parser: http_parser.o contrib/url_parser.c
$(CC) $(CPPFLAGS_FAST) $(CFLAGS_FAST) $^ -o $@
url_parser_g: http_parser_g.o contrib/url_parser.c
$(CC) $(CPPFLAGS_DEBUG) $(CFLAGS_DEBUG) $^ -o $@
parsertrace: http_parser.o contrib/parsertrace.c
$(CC) $(CPPFLAGS_FAST) $(CFLAGS_FAST) $^ -o parsertrace$(BINEXT)
parsertrace_g: http_parser_g.o contrib/parsertrace.c
$(CC) $(CPPFLAGS_DEBUG) $(CFLAGS_DEBUG) $^ -o parsertrace_g$(BINEXT)
tags: http_parser.c http_parser.h test.c
ctags $^
install: library
$(INSTALL) -D http_parser.h $(DESTDIR)$(INCLUDEDIR)/http_parser.h
$(INSTALL) -D $(LIBNAME) $(DESTDIR)$(LIBDIR)/$(LIBNAME)
ln -sf $(LIBNAME) $(DESTDIR)$(LIBDIR)/$(SONAME)
ln -sf $(LIBNAME) $(DESTDIR)$(LIBDIR)/$(SOLIBNAME).$(SOEXT)
install-strip: library
$(INSTALL) -D http_parser.h $(DESTDIR)$(INCLUDEDIR)/http_parser.h
$(INSTALL) -D -s $(LIBNAME) $(DESTDIR)$(LIBDIR)/$(LIBNAME)
ln -sf $(LIBNAME) $(DESTDIR)$(LIBDIR)/$(SONAME)
ln -sf $(LIBNAME) $(DESTDIR)$(LIBDIR)/$(SOLIBNAME).$(SOEXT)
uninstall:
rm $(DESTDIR)$(INCLUDEDIR)/http_parser.h
rm $(DESTDIR)$(LIBDIR)/$(SOLIBNAME).$(SOEXT)
rm $(DESTDIR)$(LIBDIR)/$(SONAME)
rm $(DESTDIR)$(LIBDIR)/$(LIBNAME)
clean:
rm -f *.o *.a tags test test_fast test_g \
http_parser.tar libhttp_parser.so.* \
url_parser url_parser_g parsertrace parsertrace_g \
*.exe *.exe.so
contrib/url_parser.c: http_parser.h
contrib/parsertrace.c: http_parser.h
.PHONY: clean package test-run test-run-timed test-valgrind install install-strip uninstall

249
arcs-sdk/modules/coreHTTP/source/dependency/3rdparty/http_parser/README.md vendored Normal file
View File

@@ -0,0 +1,249 @@
HTTP Parser
===========
http-parser is [**not** actively maintained](https://github.com/nodejs/http-parser/issues/522).
New projects and projects looking to migrate should consider [llhttp](https://github.com/nodejs/llhttp).
[![Build Status](https://api.travis-ci.org/nodejs/http-parser.svg?branch=master)](https://travis-ci.org/nodejs/http-parser)
This is a parser for HTTP messages written in C. It parses both requests and
responses. The parser is designed to be used in performance HTTP
applications. It does not make any syscalls nor allocations, it does not
buffer data, it can be interrupted at anytime. Depending on your
architecture, it only requires about 40 bytes of data per message
stream (in a web server that is per connection).
Features:
* No dependencies
* Handles persistent streams (keep-alive).
* Decodes chunked encoding.
* Upgrade support
* Defends against buffer overflow attacks.
The parser extracts the following information from HTTP messages:
* Header fields and values
* Content-Length
* Request method
* Response status code
* Transfer-Encoding
* HTTP version
* Request URL
* Message body
Usage
-----
One `http_parser` object is used per TCP connection. Initialize the struct
using `http_parser_init()` and set the callbacks. That might look something
like this for a request parser:
```c
http_parser_settings settings;
settings.on_url = my_url_callback;
settings.on_header_field = my_header_field_callback;
/* ... */
http_parser *parser = malloc(sizeof(http_parser));
http_parser_init(parser, HTTP_REQUEST);
parser->data = my_socket;
```
When data is received on the socket execute the parser and check for errors.
```c
size_t len = 80*1024, nparsed;
char buf[len];
ssize_t recved;
recved = recv(fd, buf, len, 0);
if (recved < 0) {
/* Handle error. */
}
/* Start up / continue the parser.
* Note we pass recved==0 to signal that EOF has been received.
*/
nparsed = http_parser_execute(parser, &settings, buf, recved);
if (parser->upgrade) {
/* handle new protocol */
} else if (nparsed != recved) {
/* Handle error. Usually just close the connection. */
}
```
`http_parser` needs to know where the end of the stream is. For example, sometimes
servers send responses without Content-Length and expect the client to
consume input (for the body) until EOF. To tell `http_parser` about EOF, give
`0` as the fourth parameter to `http_parser_execute()`. Callbacks and errors
can still be encountered during an EOF, so one must still be prepared
to receive them.
Scalar valued message information such as `status_code`, `method`, and the
HTTP version are stored in the parser structure. This data is only
temporally stored in `http_parser` and gets reset on each new message. If
this information is needed later, copy it out of the structure during the
`headers_complete` callback.
The parser decodes the transfer-encoding for both requests and responses
transparently. That is, a chunked encoding is decoded before being sent to
the on_body callback.
The Special Problem of Upgrade
------------------------------
`http_parser` supports upgrading the connection to a different protocol. An
increasingly common example of this is the WebSocket protocol which sends
a request like
GET /demo HTTP/1.1
Upgrade: WebSocket
Connection: Upgrade
Host: example.com
Origin: http://example.com
WebSocket-Protocol: sample
followed by non-HTTP data.
(See [RFC6455](https://tools.ietf.org/html/rfc6455) for more information the
WebSocket protocol.)
To support this, the parser will treat this as a normal HTTP message without a
body, issuing both on_headers_complete and on_message_complete callbacks. However
http_parser_execute() will stop parsing at the end of the headers and return.
The user is expected to check if `parser->upgrade` has been set to 1 after
`http_parser_execute()` returns. Non-HTTP data begins at the buffer supplied
offset by the return value of `http_parser_execute()`.
Callbacks
---------
During the `http_parser_execute()` call, the callbacks set in
`http_parser_settings` will be executed. The parser maintains state and
never looks behind, so buffering the data is not necessary. If you need to
save certain data for later usage, you can do that from the callbacks.
There are two types of callbacks:
* notification `typedef int (*http_cb) (http_parser*);`
Callbacks: on_message_begin, on_headers_complete, on_message_complete.
* data `typedef int (*http_data_cb) (http_parser*, const char *at, size_t length);`
Callbacks: (requests only) on_url,
(common) on_header_field, on_header_value, on_body;
Callbacks must return 0 on success. Returning a non-zero value indicates
error to the parser, making it exit immediately.
For cases where it is necessary to pass local information to/from a callback,
the `http_parser` object's `data` field can be used.
An example of such a case is when using threads to handle a socket connection,
parse a request, and then give a response over that socket. By instantiation
of a thread-local struct containing relevant data (e.g. accepted socket,
allocated memory for callbacks to write into, etc), a parser's callbacks are
able to communicate data between the scope of the thread and the scope of the
callback in a threadsafe manner. This allows `http_parser` to be used in
multi-threaded contexts.
Example:
```c
typedef struct {
socket_t sock;
void* buffer;
int buf_len;
} custom_data_t;
int my_url_callback(http_parser* parser, const char *at, size_t length) {
/* access to thread local custom_data_t struct.
Use this access save parsed data for later use into thread local
buffer, or communicate over socket
*/
parser->data;
...
return 0;
}
...
void http_parser_thread(socket_t sock) {
int nparsed = 0;
/* allocate memory for user data */
custom_data_t *my_data = malloc(sizeof(custom_data_t));
/* some information for use by callbacks.
* achieves thread -> callback information flow */
my_data->sock = sock;
/* instantiate a thread-local parser */
http_parser *parser = malloc(sizeof(http_parser));
http_parser_init(parser, HTTP_REQUEST); /* initialise parser */
/* this custom data reference is accessible through the reference to the
parser supplied to callback functions */
parser->data = my_data;
http_parser_settings settings; /* set up callbacks */
settings.on_url = my_url_callback;
/* execute parser */
nparsed = http_parser_execute(parser, &settings, buf, recved);
...
/* parsed information copied from callback.
can now perform action on data copied into thread-local memory from callbacks.
achieves callback -> thread information flow */
my_data->buffer;
...
}
```
In case you parse HTTP message in chunks (i.e. `read()` request line
from socket, parse, read half headers, parse, etc) your data callbacks
may be called more than once. `http_parser` guarantees that data pointer is only
valid for the lifetime of callback. You can also `read()` into a heap allocated
buffer to avoid copying memory around if this fits your application.
Reading headers may be a tricky task if you read/parse headers partially.
Basically, you need to remember whether last header callback was field or value
and apply the following logic:
(on_header_field and on_header_value shortened to on_h_*)
------------------------ ------------ --------------------------------------------
| State (prev. callback) | Callback | Description/action |
------------------------ ------------ --------------------------------------------
| nothing (first call) | on_h_field | Allocate new buffer and copy callback data |
| | | into it |
------------------------ ------------ --------------------------------------------
| value | on_h_field | New header started. |
| | | Copy current name,value buffers to headers |
| | | list and allocate new buffer for new name |
------------------------ ------------ --------------------------------------------
| field | on_h_field | Previous name continues. Reallocate name |
| | | buffer and append callback data to it |
------------------------ ------------ --------------------------------------------
| field | on_h_value | Value for current header started. Allocate |
| | | new buffer and copy callback data to it |
------------------------ ------------ --------------------------------------------
| value | on_h_value | Value continues. Reallocate value buffer |
| | | and append callback data to it |
------------------------ ------------ --------------------------------------------
Parsing URLs
------------
A simplistic zero-copy URL parser is provided as `http_parser_parse_url()`.
Users of this library may wish to use it to parse URLs constructed from
consecutive `on_url` callbacks.
See examples of reading in headers:
* [partial example](http://gist.github.com/155877) in C
* [from http-parser tests](http://github.com/joyent/http-parser/blob/37a0ff8/test.c#L403) in C
* [from Node library](http://github.com/joyent/node/blob/842eaf4/src/http.js#L284) in Javascript

128
arcs-sdk/modules/coreHTTP/source/dependency/3rdparty/http_parser/bench.c vendored Normal file
View File

@@ -0,0 +1,128 @@
/* Copyright Fedor Indutny. All rights reserved.
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to
* deal in the Software without restriction, including without limitation the
* rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
* sell copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
* IN THE SOFTWARE.
*/
#include "http_parser.h"
#include <assert.h>
#include <stdint.h>
#include <stdio.h>
#include <string.h>
#include <sys/time.h>
/* 8 gb */
static const int64_t kBytes = 8LL << 30;
static const char data[] =
"POST /joyent/http-parser HTTP/1.1\r\n"
"Host: github.com\r\n"
"DNT: 1\r\n"
"Accept-Encoding: gzip, deflate, sdch\r\n"
"Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.6,en;q=0.4\r\n"
"User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) "
"AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/39.0.2171.65 Safari/537.36\r\n"
"Accept: text/html,application/xhtml+xml,application/xml;q=0.9,"
"image/webp,*/*;q=0.8\r\n"
"Referer: https://github.com/joyent/http-parser\r\n"
"Connection: keep-alive\r\n"
"Transfer-Encoding: chunked\r\n"
"Cache-Control: max-age=0\r\n\r\nb\r\nhello world\r\n0\r\n";
static const size_t data_len = sizeof(data) - 1;
static int on_info(http_parser* p) {
return 0;
}
static int on_data(http_parser* p, const char *at, size_t length) {
return 0;
}
static http_parser_settings settings = {
.on_message_begin = on_info,
.on_headers_complete = on_info,
.on_message_complete = on_info,
.on_header_field = on_data,
.on_header_value = on_data,
.on_url = on_data,
.on_status = on_data,
.on_body = on_data
};
int bench(int iter_count, int silent) {
struct http_parser parser;
int i;
int err;
struct timeval start;
struct timeval end;
if (!silent) {
err = gettimeofday(&start, NULL);
assert(err == 0);
}
fprintf(stderr, "req_len=%d\n", (int) data_len);
for (i = 0; i < iter_count; i++) {
size_t parsed;
http_parser_init(&parser, HTTP_REQUEST);
parsed = http_parser_execute(&parser, &settings, data, data_len);
assert(parsed == data_len);
}
if (!silent) {
double elapsed;
double bw;
double total;
err = gettimeofday(&end, NULL);
assert(err == 0);
fprintf(stdout, "Benchmark result:\n");
elapsed = (double) (end.tv_sec - start.tv_sec) +
(end.tv_usec - start.tv_usec) * 1e-6f;
total = (double) iter_count * data_len;
bw = (double) total / elapsed;
fprintf(stdout, "%.2f mb | %.2f mb/s | %.2f req/sec | %.2f s\n",
(double) total / (1024 * 1024),
bw / (1024 * 1024),
(double) iter_count / elapsed,
elapsed);
fflush(stdout);
}
return 0;
}
int main(int argc, char** argv) {
int64_t iterations;
iterations = kBytes / (int64_t) data_len;
if (argc == 2 && strcmp(argv[1], "infinite") == 0) {
for (;;)
bench(iterations, 1);
return 0;
} else {
return bench(iterations, 0);
}
}

157
arcs-sdk/modules/coreHTTP/source/dependency/3rdparty/http_parser/contrib/parsertrace.c vendored Normal file
View File

@@ -0,0 +1,157 @@
/* Copyright Joyent, Inc. and other Node contributors.
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to
* deal in the Software without restriction, including without limitation the
* rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
* sell copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
* IN THE SOFTWARE.
*/
/* Dump what the parser finds to stdout as it happen */
#include "http_parser.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int on_message_begin(http_parser* _) {
(void)_;
printf("\n***MESSAGE BEGIN***\n\n");
return 0;
}
int on_headers_complete(http_parser* _) {
(void)_;
printf("\n***HEADERS COMPLETE***\n\n");
return 0;
}
int on_message_complete(http_parser* _) {
(void)_;
printf("\n***MESSAGE COMPLETE***\n\n");
return 0;
}
int on_url(http_parser* _, const char* at, size_t length) {
(void)_;
printf("Url: %.*s\n", (int)length, at);
return 0;
}
int on_header_field(http_parser* _, const char* at, size_t length) {
(void)_;
printf("Header field: %.*s\n", (int)length, at);
return 0;
}
int on_header_value(http_parser* _, const char* at, size_t length) {
(void)_;
printf("Header value: %.*s\n", (int)length, at);
return 0;
}
int on_body(http_parser* _, const char* at, size_t length) {
(void)_;
printf("Body: %.*s\n", (int)length, at);
return 0;
}
void usage(const char* name) {
fprintf(stderr,
"Usage: %s $type $filename\n"
" type: -x, where x is one of {r,b,q}\n"
" parses file as a Response, reQuest, or Both\n",
name);
exit(EXIT_FAILURE);
}
int main(int argc, char* argv[]) {
enum http_parser_type file_type;
if (argc != 3) {
usage(argv[0]);
}
char* type = argv[1];
if (type[0] != '-') {
usage(argv[0]);
}
switch (type[1]) {
/* in the case of "-", type[1] will be NUL */
case 'r':
file_type = HTTP_RESPONSE;
break;
case 'q':
file_type = HTTP_REQUEST;
break;
case 'b':
file_type = HTTP_BOTH;
break;
default:
usage(argv[0]);
}
char* filename = argv[2];
FILE* file = fopen(filename, "r");
if (file == NULL) {
perror("fopen");
goto fail;
}
fseek(file, 0, SEEK_END);
long file_length = ftell(file);
if (file_length == -1) {
perror("ftell");
goto fail;
}
fseek(file, 0, SEEK_SET);
char* data = malloc(file_length);
if (fread(data, 1, file_length, file) != (size_t)file_length) {
fprintf(stderr, "couldn't read entire file\n");
free(data);
goto fail;
}
http_parser_settings settings;
memset(&settings, 0, sizeof(settings));
settings.on_message_begin = on_message_begin;
settings.on_url = on_url;
settings.on_header_field = on_header_field;
settings.on_header_value = on_header_value;
settings.on_headers_complete = on_headers_complete;
settings.on_body = on_body;
settings.on_message_complete = on_message_complete;
http_parser parser;
http_parser_init(&parser, file_type);
size_t nparsed = http_parser_execute(&parser, &settings, data, file_length);
free(data);
if (nparsed != (size_t)file_length) {
fprintf(stderr,
"Error: %s (%s)\n",
http_errno_description(HTTP_PARSER_ERRNO(&parser)),
http_errno_name(HTTP_PARSER_ERRNO(&parser)));
goto fail;
}
return EXIT_SUCCESS;
fail:
fclose(file);
return EXIT_FAILURE;
}

47
arcs-sdk/modules/coreHTTP/source/dependency/3rdparty/http_parser/contrib/url_parser.c vendored Normal file
View File

@@ -0,0 +1,47 @@
#include "http_parser.h"
#include <stdio.h>
#include <string.h>
void
dump_url (const char *url, const struct http_parser_url *u)
{
unsigned int i;
printf("\tfield_set: 0x%x, port: %u\n", u->field_set, u->port);
for (i = 0; i < UF_MAX; i++) {
if ((u->field_set & (1 << i)) == 0) {
printf("\tfield_data[%u]: unset\n", i);
continue;
}
printf("\tfield_data[%u]: off: %u, len: %u, part: %.*s\n",
i,
u->field_data[i].off,
u->field_data[i].len,
u->field_data[i].len,
url + u->field_data[i].off);
}
}
int main(int argc, char ** argv) {
struct http_parser_url u;
int len, connect, result;
if (argc != 3) {
printf("Syntax : %s connect|get url\n", argv[0]);
return 1;
}
len = strlen(argv[2]);
connect = strcmp("connect", argv[1]) == 0 ? 1 : 0;
printf("Parsing %s, connect %d\n", argv[2], connect);
http_parser_url_init(&u);
result = http_parser_parse_url(argv[2], len, connect, &u);
if (result != 0) {
printf("Parse error : %d\n", result);
return result;
}
printf("Parse ok, result : \n");
dump_url(argv[2], &u);
return 0;
}

26
arcs-sdk/modules/coreHTTP/source/dependency/3rdparty/http_parser/fuzzers/fuzz_parser.c vendored Normal file
View File

@@ -0,0 +1,26 @@
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include "http_parser.h"
int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
static const http_parser_settings settings_null = {
.on_message_begin = 0
, .on_header_field = 0
,.on_header_value = 0
,.on_url = 0
,.on_status = 0
,.on_body = 0
,.on_headers_complete = 0
,.on_message_complete = 0
,.on_chunk_header = 0
,.on_chunk_complete = 0
};
http_parser parser;
http_parser_init(&parser, HTTP_BOTH);
http_parser_execute(&parser, &settings_null, (char*)data, size);
return 0;
}

14
arcs-sdk/modules/coreHTTP/source/dependency/3rdparty/http_parser/fuzzers/fuzz_url.c vendored Normal file
View File

@@ -0,0 +1,14 @@
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include "http_parser.h"
int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
struct http_parser_url u;
http_parser_url_init(&u);
http_parser_parse_url((char*)data, size, 0, &u);
http_parser_parse_url((char*)data, size, 1, &u);
return 0;
}

2575
arcs-sdk/modules/coreHTTP/source/dependency/3rdparty/http_parser/http_parser.c vendored Normal file
View File

File diff suppressed because it is too large Load Diff

111
arcs-sdk/modules/coreHTTP/source/dependency/3rdparty/http_parser/http_parser.gyp vendored Normal file
View File

@@ -0,0 +1,111 @@
# This file is used with the GYP meta build system.
# http://code.google.com/p/gyp/
# To build try this:
# svn co http://gyp.googlecode.com/svn/trunk gyp
# ./gyp/gyp -f make --depth=`pwd` http_parser.gyp
# ./out/Debug/test
{
'target_defaults': {
'default_configuration': 'Debug',
'configurations': {
# TODO: hoist these out and put them somewhere common, because
# RuntimeLibrary MUST MATCH across the entire project
'Debug': {
'defines': [ 'DEBUG', '_DEBUG' ],
'cflags': [ '-Wall', '-Wextra', '-O0', '-g', '-ftrapv' ],
'msvs_settings': {
'VCCLCompilerTool': {
'RuntimeLibrary': 1, # static debug
},
},
},
'Release': {
'defines': [ 'NDEBUG' ],
'cflags': [ '-Wall', '-Wextra', '-O3' ],
'msvs_settings': {
'VCCLCompilerTool': {
'RuntimeLibrary': 0, # static release
},
},
}
},
'msvs_settings': {
'VCCLCompilerTool': {
},
'VCLibrarianTool': {
},
'VCLinkerTool': {
'GenerateDebugInformation': 'true',
},
},
'conditions': [
['OS == "win"', {
'defines': [
'WIN32'
],
}]
],
},
'targets': [
{
'target_name': 'http_parser',
'type': 'static_library',
'include_dirs': [ '.' ],
'direct_dependent_settings': {
'defines': [ 'HTTP_PARSER_STRICT=0' ],
'include_dirs': [ '.' ],
},
'defines': [ 'HTTP_PARSER_STRICT=0' ],
'sources': [ './http_parser.c', ],
'conditions': [
['OS=="win"', {
'msvs_settings': {
'VCCLCompilerTool': {
# Compile as C++. http_parser.c is actually C99, but C++ is
# close enough in this case.
'CompileAs': 2,
},
},
}]
],
},
{
'target_name': 'http_parser_strict',
'type': 'static_library',
'include_dirs': [ '.' ],
'direct_dependent_settings': {
'defines': [ 'HTTP_PARSER_STRICT=1' ],
'include_dirs': [ '.' ],
},
'defines': [ 'HTTP_PARSER_STRICT=1' ],
'sources': [ './http_parser.c', ],
'conditions': [
['OS=="win"', {
'msvs_settings': {
'VCCLCompilerTool': {
# Compile as C++. http_parser.c is actually C99, but C++ is
# close enough in this case.
'CompileAs': 2,
},
},
}]
],
},
{
'target_name': 'test-nonstrict',
'type': 'executable',
'dependencies': [ 'http_parser' ],
'sources': [ 'test.c' ]
},
{
'target_name': 'test-strict',
'type': 'executable',
'dependencies': [ 'http_parser_strict' ],
'sources': [ 'test.c' ]
}
]
}

449
arcs-sdk/modules/coreHTTP/source/dependency/3rdparty/http_parser/http_parser.h vendored Normal file
View File

@@ -0,0 +1,449 @@
/* Copyright Joyent, Inc. and other Node contributors. All rights reserved.
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to
* deal in the Software without restriction, including without limitation the
* rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
* sell copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
* IN THE SOFTWARE.
*/
#ifndef http_parser_h
#define http_parser_h
#ifdef __cplusplus
extern "C" {
#endif
/* Also update SONAME in the Makefile whenever you change these. */
#define HTTP_PARSER_VERSION_MAJOR 2
#define HTTP_PARSER_VERSION_MINOR 9
#define HTTP_PARSER_VERSION_PATCH 4
#include <stddef.h>
#if defined(_WIN32) && !defined(__MINGW32__) && \
(!defined(_MSC_VER) || _MSC_VER<1600) && !defined(__WINE__)
#include <BaseTsd.h>
typedef __int8 int8_t;
typedef unsigned __int8 uint8_t;
typedef __int16 int16_t;
typedef unsigned __int16 uint16_t;
typedef __int32 int32_t;
typedef unsigned __int32 uint32_t;
typedef __int64 int64_t;
typedef unsigned __int64 uint64_t;
#elif (defined(__sun) || defined(__sun__)) && defined(__SunOS_5_9)
#include <sys/inttypes.h>
#else
#include <stdint.h>
#endif
/* Compile with -DHTTP_PARSER_STRICT=0 to make less checks, but run
* faster
*/
#ifndef HTTP_PARSER_STRICT
# define HTTP_PARSER_STRICT 1
#endif
/* Maximium header size allowed. If the macro is not defined
* before including this header then the default is used. To
* change the maximum header size, define the macro in the build
* environment (e.g. -DHTTP_MAX_HEADER_SIZE=<value>). To remove
* the effective limit on the size of the header, define the macro
* to a very large number (e.g. -DHTTP_MAX_HEADER_SIZE=0x7fffffff)
*/
#ifndef HTTP_MAX_HEADER_SIZE
# define HTTP_MAX_HEADER_SIZE (80*1024)
#endif
typedef struct http_parser http_parser;
typedef struct http_parser_settings http_parser_settings;
/* Callbacks should return non-zero to indicate an error. The parser will
* then halt execution.
*
* The one exception is on_headers_complete. In a HTTP_RESPONSE parser
* returning '1' from on_headers_complete will tell the parser that it
* should not expect a body. This is used when receiving a response to a
* HEAD request which may contain 'Content-Length' or 'Transfer-Encoding:
* chunked' headers that indicate the presence of a body.
*
* Returning `2` from on_headers_complete will tell parser that it should not
* expect neither a body nor any futher responses on this connection. This is
* useful for handling responses to a CONNECT request which may not contain
* `Upgrade` or `Connection: upgrade` headers.
*
* http_data_cb does not return data chunks. It will be called arbitrarily
* many times for each string. E.G. you might get 10 callbacks for "on_url"
* each providing just a few characters more data.
*/
typedef int (*http_data_cb) (http_parser*, const char *at, size_t length);
typedef int (*http_cb) (http_parser*);
/* Status Codes */
#define HTTP_STATUS_MAP(XX) \
XX(100, CONTINUE, Continue) \
XX(101, SWITCHING_PROTOCOLS, Switching Protocols) \
XX(102, PROCESSING, Processing) \
XX(200, OK, OK) \
XX(201, CREATED, Created) \
XX(202, ACCEPTED, Accepted) \
XX(203, NON_AUTHORITATIVE_INFORMATION, Non-Authoritative Information) \
XX(204, NO_CONTENT, No Content) \
XX(205, RESET_CONTENT, Reset Content) \
XX(206, PARTIAL_CONTENT, Partial Content) \
XX(207, MULTI_STATUS, Multi-Status) \
XX(208, ALREADY_REPORTED, Already Reported) \
XX(226, IM_USED, IM Used) \
XX(300, MULTIPLE_CHOICES, Multiple Choices) \
XX(301, MOVED_PERMANENTLY, Moved Permanently) \
XX(302, FOUND, Found) \
XX(303, SEE_OTHER, See Other) \
XX(304, NOT_MODIFIED, Not Modified) \
XX(305, USE_PROXY, Use Proxy) \
XX(307, TEMPORARY_REDIRECT, Temporary Redirect) \
XX(308, PERMANENT_REDIRECT, Permanent Redirect) \
XX(400, BAD_REQUEST, Bad Request) \
XX(401, UNAUTHORIZED, Unauthorized) \
XX(402, PAYMENT_REQUIRED, Payment Required) \
XX(403, FORBIDDEN, Forbidden) \
XX(404, NOT_FOUND, Not Found) \
XX(405, METHOD_NOT_ALLOWED, Method Not Allowed) \
XX(406, NOT_ACCEPTABLE, Not Acceptable) \
XX(407, PROXY_AUTHENTICATION_REQUIRED, Proxy Authentication Required) \
XX(408, REQUEST_TIMEOUT, Request Timeout) \
XX(409, CONFLICT, Conflict) \
XX(410, GONE, Gone) \
XX(411, LENGTH_REQUIRED, Length Required) \
XX(412, PRECONDITION_FAILED, Precondition Failed) \
XX(413, PAYLOAD_TOO_LARGE, Payload Too Large) \
XX(414, URI_TOO_LONG, URI Too Long) \
XX(415, UNSUPPORTED_MEDIA_TYPE, Unsupported Media Type) \
XX(416, RANGE_NOT_SATISFIABLE, Range Not Satisfiable) \
XX(417, EXPECTATION_FAILED, Expectation Failed) \
XX(421, MISDIRECTED_REQUEST, Misdirected Request) \
XX(422, UNPROCESSABLE_ENTITY, Unprocessable Entity) \
XX(423, LOCKED, Locked) \
XX(424, FAILED_DEPENDENCY, Failed Dependency) \
XX(426, UPGRADE_REQUIRED, Upgrade Required) \
XX(428, PRECONDITION_REQUIRED, Precondition Required) \
XX(429, TOO_MANY_REQUESTS, Too Many Requests) \
XX(431, REQUEST_HEADER_FIELDS_TOO_LARGE, Request Header Fields Too Large) \
XX(451, UNAVAILABLE_FOR_LEGAL_REASONS, Unavailable For Legal Reasons) \
XX(500, INTERNAL_SERVER_ERROR, Internal Server Error) \
XX(501, NOT_IMPLEMENTED, Not Implemented) \
XX(502, BAD_GATEWAY, Bad Gateway) \
XX(503, SERVICE_UNAVAILABLE, Service Unavailable) \
XX(504, GATEWAY_TIMEOUT, Gateway Timeout) \
XX(505, HTTP_VERSION_NOT_SUPPORTED, HTTP Version Not Supported) \
XX(506, VARIANT_ALSO_NEGOTIATES, Variant Also Negotiates) \
XX(507, INSUFFICIENT_STORAGE, Insufficient Storage) \
XX(508, LOOP_DETECTED, Loop Detected) \
XX(510, NOT_EXTENDED, Not Extended) \
XX(511, NETWORK_AUTHENTICATION_REQUIRED, Network Authentication Required) \
enum http_status
{
#define XX(num, name, string) HTTP_STATUS_##name = num,
HTTP_STATUS_MAP(XX)
#undef XX
};
/* Request Methods */
#define HTTP_METHOD_MAP(XX) \
XX(0, DELETE, DELETE) \
XX(1, GET, GET) \
XX(2, HEAD, HEAD) \
XX(3, POST, POST) \
XX(4, PUT, PUT) \
/* pathological */ \
XX(5, CONNECT, CONNECT) \
XX(6, OPTIONS, OPTIONS) \
XX(7, TRACE, TRACE) \
/* WebDAV */ \
XX(8, COPY, COPY) \
XX(9, LOCK, LOCK) \
XX(10, MKCOL, MKCOL) \
XX(11, MOVE, MOVE) \
XX(12, PROPFIND, PROPFIND) \
XX(13, PROPPATCH, PROPPATCH) \
XX(14, SEARCH, SEARCH) \
XX(15, UNLOCK, UNLOCK) \
XX(16, BIND, BIND) \
XX(17, REBIND, REBIND) \
XX(18, UNBIND, UNBIND) \
XX(19, ACL, ACL) \
/* subversion */ \
XX(20, REPORT, REPORT) \
XX(21, MKACTIVITY, MKACTIVITY) \
XX(22, CHECKOUT, CHECKOUT) \
XX(23, MERGE, MERGE) \
/* upnp */ \
XX(24, MSEARCH, M-SEARCH) \
XX(25, NOTIFY, NOTIFY) \
XX(26, SUBSCRIBE, SUBSCRIBE) \
XX(27, UNSUBSCRIBE, UNSUBSCRIBE) \
/* RFC-5789 */ \
XX(28, PATCH, PATCH) \
XX(29, PURGE, PURGE) \
/* CalDAV */ \
XX(30, MKCALENDAR, MKCALENDAR) \
/* RFC-2068, section 19.6.1.2 */ \
XX(31, LINK, LINK) \
XX(32, UNLINK, UNLINK) \
/* icecast */ \
XX(33, SOURCE, SOURCE) \
enum http_method
{
#define XX(num, name, string) HTTP_##name = num,
HTTP_METHOD_MAP(XX)
#undef XX
};
enum http_parser_type { HTTP_REQUEST, HTTP_RESPONSE, HTTP_BOTH };
/* Flag values for http_parser.flags field */
enum flags
{ F_CHUNKED = 1 << 0
, F_CONNECTION_KEEP_ALIVE = 1 << 1
, F_CONNECTION_CLOSE = 1 << 2
, F_CONNECTION_UPGRADE = 1 << 3
, F_TRAILING = 1 << 4
, F_UPGRADE = 1 << 5
, F_SKIPBODY = 1 << 6
, F_CONTENTLENGTH = 1 << 7
};
/* Map for errno-related constants
*
* The provided argument should be a macro that takes 2 arguments.
*/
#define HTTP_ERRNO_MAP(XX) \
/* No error */ \
XX(OK, "success") \
\
/* Callback-related errors */ \
XX(CB_message_begin, "the on_message_begin callback failed") \
XX(CB_url, "the on_url callback failed") \
XX(CB_header_field, "the on_header_field callback failed") \
XX(CB_header_value, "the on_header_value callback failed") \
XX(CB_headers_complete, "the on_headers_complete callback failed") \
XX(CB_body, "the on_body callback failed") \
XX(CB_message_complete, "the on_message_complete callback failed") \
XX(CB_status, "the on_status callback failed") \
XX(CB_chunk_header, "the on_chunk_header callback failed") \
XX(CB_chunk_complete, "the on_chunk_complete callback failed") \
\
/* Parsing-related errors */ \
XX(INVALID_EOF_STATE, "stream ended at an unexpected time") \
XX(HEADER_OVERFLOW, \
"too many header bytes seen; overflow detected") \
XX(CLOSED_CONNECTION, \
"data received after completed connection: close message") \
XX(INVALID_VERSION, "invalid HTTP version") \
XX(INVALID_STATUS, "invalid HTTP status code") \
XX(INVALID_METHOD, "invalid HTTP method") \
XX(INVALID_URL, "invalid URL") \
XX(INVALID_HOST, "invalid host") \
XX(INVALID_PORT, "invalid port") \
XX(INVALID_PATH, "invalid path") \
XX(INVALID_QUERY_STRING, "invalid query string") \
XX(INVALID_FRAGMENT, "invalid fragment") \
XX(LF_EXPECTED, "LF character expected") \
XX(INVALID_HEADER_TOKEN, "invalid character in header") \
XX(INVALID_CONTENT_LENGTH, \
"invalid character in content-length header") \
XX(UNEXPECTED_CONTENT_LENGTH, \
"unexpected content-length header") \
XX(INVALID_CHUNK_SIZE, \
"invalid character in chunk size header") \
XX(INVALID_CONSTANT, "invalid constant string") \
XX(INVALID_INTERNAL_STATE, "encountered unexpected internal state")\
XX(STRICT, "strict mode assertion failed") \
XX(PAUSED, "parser is paused") \
XX(UNKNOWN, "an unknown error occurred") \
XX(INVALID_TRANSFER_ENCODING, \
"request has invalid transfer-encoding") \
/* Define HPE_* values for each errno value above */
#define HTTP_ERRNO_GEN(n, s) HPE_##n,
enum http_errno {
HTTP_ERRNO_MAP(HTTP_ERRNO_GEN)
};
#undef HTTP_ERRNO_GEN
/* Get an http_errno value from an http_parser */
#define HTTP_PARSER_ERRNO(p) ((enum http_errno) (p)->http_errno)
struct http_parser {
/** PRIVATE **/
unsigned int type : 2; /* enum http_parser_type */
unsigned int flags : 8; /* F_* values from 'flags' enum; semi-public */
unsigned int state : 7; /* enum state from http_parser.c */
unsigned int header_state : 7; /* enum header_state from http_parser.c */
unsigned int index : 5; /* index into current matcher */
unsigned int uses_transfer_encoding : 1; /* Transfer-Encoding header is present */
unsigned int allow_chunked_length : 1; /* Allow headers with both
* `Content-Length` and
* `Transfer-Encoding: chunked` set */
unsigned int lenient_http_headers : 1;
uint32_t nread; /* # bytes read in various scenarios */
uint64_t content_length; /* # bytes in body. `(uint64_t) -1` (all bits one)
* if no Content-Length header.
*/
/** READ-ONLY **/
unsigned short http_major;
unsigned short http_minor;
unsigned int status_code : 16; /* responses only */
unsigned int method : 8; /* requests only */
unsigned int http_errno : 7;
/* 1 = Upgrade header was present and the parser has exited because of that.
* 0 = No upgrade header present.
* Should be checked when http_parser_execute() returns in addition to
* error checking.
*/
unsigned int upgrade : 1;
/** PUBLIC **/
void *data; /* A pointer to get hook to the "connection" or "socket" object */
};
struct http_parser_settings {
http_cb on_message_begin;
http_data_cb on_url;
http_data_cb on_status;
http_data_cb on_header_field;
http_data_cb on_header_value;
http_cb on_headers_complete;
http_data_cb on_body;
http_cb on_message_complete;
/* When on_chunk_header is called, the current chunk length is stored
* in parser->content_length.
*/
http_cb on_chunk_header;
http_cb on_chunk_complete;
};
enum http_parser_url_fields
{ UF_SCHEMA = 0
, UF_HOST = 1
, UF_PORT = 2
, UF_PATH = 3
, UF_QUERY = 4
, UF_FRAGMENT = 5
, UF_USERINFO = 6
, UF_MAX = 7
};
/* Result structure for http_parser_parse_url().
*
* Callers should index into field_data[] with UF_* values iff field_set
* has the relevant (1 << UF_*) bit set. As a courtesy to clients (and
* because we probably have padding left over), we convert any port to
* a uint16_t.
*/
struct http_parser_url {
uint16_t field_set; /* Bitmask of (1 << UF_*) values */
uint16_t port; /* Converted UF_PORT string */
struct {
uint16_t off; /* Offset into buffer in which field starts */
uint16_t len; /* Length of run in buffer */
} field_data[UF_MAX];
};
/* Returns the library version. Bits 16-23 contain the major version number,
* bits 8-15 the minor version number and bits 0-7 the patch level.
* Usage example:
*
* unsigned long version = http_parser_version();
* unsigned major = (version >> 16) & 255;
* unsigned minor = (version >> 8) & 255;
* unsigned patch = version & 255;
* printf("http_parser v%u.%u.%u\n", major, minor, patch);
*/
unsigned long http_parser_version(void);
void http_parser_init(http_parser *parser, enum http_parser_type type);
/* Initialize http_parser_settings members to 0
*/
void http_parser_settings_init(http_parser_settings *settings);
/* Executes the parser. Returns number of parsed bytes. Sets
* `parser->http_errno` on error. */
size_t http_parser_execute(http_parser *parser,
const http_parser_settings *settings,
const char *data,
size_t len);
/* If http_should_keep_alive() in the on_headers_complete or
* on_message_complete callback returns 0, then this should be
* the last message on the connection.
* If you are the server, respond with the "Connection: close" header.
* If you are the client, close the connection.
*/
int http_should_keep_alive(const http_parser *parser);
/* Returns a string version of the HTTP method. */
const char *http_method_str(enum http_method m);
/* Returns a string version of the HTTP status code. */
const char *http_status_str(enum http_status s);
/* Return a string name of the given error */
const char *http_errno_name(enum http_errno err);
/* Return a string description of the given error */
const char *http_errno_description(enum http_errno err);
/* Initialize all http_parser_url members to 0 */
void http_parser_url_init(struct http_parser_url *u);
/* Parse a URL; return nonzero on failure */
int http_parser_parse_url(const char *buf, size_t buflen,
int is_connect,
struct http_parser_url *u);
/* Pause or un-pause the parser; a nonzero value pauses */
void http_parser_pause(http_parser *parser, int paused);
/* Checks if this is the final chunk of the body. */
int http_body_is_final(const http_parser *parser);
/* Change the maximum header size provided at compile time. */
void http_parser_set_max_header_size(uint32_t size);
#ifdef __cplusplus
}
#endif
#endif

4729
arcs-sdk/modules/coreHTTP/source/dependency/3rdparty/http_parser/test.c vendored Normal file
View File

File diff suppressed because it is too large Load Diff

919
arcs-sdk/modules/coreHTTP/source/include/core_http_client.h Normal file
View File

@@ -0,0 +1,919 @@
/*
* coreHTTP v2.1.0
* Copyright (C) 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of
* this software and associated documentation files (the "Software"), to deal in
* the Software without restriction, including without limitation the rights to
* use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
* the Software, and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
* IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
/**
* @file core_http_client.h
* @brief User facing functions of the HTTP Client library.
*/
#ifndef CORE_HTTP_CLIENT_H_
#define CORE_HTTP_CLIENT_H_
#include <stdint.h>
#include <stddef.h>
/* *INDENT-OFF* */
#ifdef __cplusplus
extern "C" {
#endif
/* *INDENT-ON* */
/* HTTP_DO_NOT_USE_CUSTOM_CONFIG allows building the HTTP Client library
* without a config file. If a config file is provided, the
* HTTP_DO_NOT_USE_CUSTOM_CONFIG macro must not be defined.
*/
#ifndef HTTP_DO_NOT_USE_CUSTOM_CONFIG
#include "core_http_config.h"
#endif
/* Include config defaults header to get default values of configurations not
* defined in core_http_config.h file. */
#include "core_http_config_defaults.h"
/* Transport interface include. */
#include "transport_interface.h"
/* Convenience macros for some HTTP request methods. */
/** @addtogroup http_constants
* @{
*/
#define HTTP_METHOD_GET "GET" /**< HTTP Method GET string. */
#define HTTP_METHOD_PUT "PUT" /**< HTTP Method PUT string. */
#define HTTP_METHOD_POST "POST" /**< HTTP Method POST string. */
#define HTTP_METHOD_HEAD "HEAD" /**< HTTP Method HEAD string. */
/** @}*/
/**
* @ingroup http_constants
* @brief The maximum Content-Length header field and value that could be
* written to the request header buffer.
*/
#define HTTP_MAX_CONTENT_LENGTH_HEADER_LENGTH sizeof( "Content-Length: 4294967295" ) - 1U
/**
* @defgroup http_send_flags HTTPClient_Send Flags
* @brief Values for #HTTPClient_Send sendFlags parameter.
* These flags control some behavior of sending the request or receiving the
* response.
*
* Flags should be bitwise-ORed with each other to change the behavior of
* #HTTPClient_Send.
*/
/**
* @ingroup http_send_flags
* @brief Set this flag to disable automatically writing the Content-Length
* header to send to the server.
*
* This flag is valid only for #HTTPClient_Send sendFlags parameter.
*/
#define HTTP_SEND_DISABLE_CONTENT_LENGTH_FLAG 0x1U
/**
* @defgroup http_request_flags HTTPRequestInfo_t Flags
* @brief Flags for #HTTPRequestInfo_t.reqFlags.
* These flags control what headers are written or not to the
* #HTTPRequestHeaders_t.pBuffer by #HTTPClient_InitializeRequestHeaders.
*
* Flags should be bitwise-ORed with each other to change the behavior of
* #HTTPClient_InitializeRequestHeaders.
*/
/**
* @ingroup http_request_flags
* @brief Set this flag to indicate that the request is for a persistent
* connection.
*
* Setting this will cause a "Connection: Keep-Alive" to be written to the
* request headers.
*
* This flag is valid only for #HTTPRequestInfo_t reqFlags parameter.
*/
#define HTTP_REQUEST_KEEP_ALIVE_FLAG 0x1U
/**
* @defgroup http_response_flags HTTPResponse_t Flags
* @brief Flags for #HTTPResponse_t.respFlags.
* These flags are populated in #HTTPResponse_t.respFlags by the #HTTPClient_Send
* function.
*
* A flag's value can be extracted from #HTTPResponse_t.respFlags with a
* bitwise-AND.
*/
/**
* @ingroup http_response_flags
* @brief This will be set to true if header "Connection: close" is found.
*
* If a "Connection: close" header is present the application should always
* close the connection.
*
* This flag is valid only for #HTTPResponse_t.respFlags.
*/
#define HTTP_RESPONSE_CONNECTION_CLOSE_FLAG 0x1U
/**
* @ingroup http_response_flags
* @brief This will be set to true if header "Connection: Keep-Alive" is found.
*
* This flag is valid only for #HTTPResponse_t.respFlags.
*/
#define HTTP_RESPONSE_CONNECTION_KEEP_ALIVE_FLAG 0x2U
/**
* @ingroup http_constants
* @brief Flag that represents End of File byte in the range specification of
* a Range Request.
* This flag should be used ONLY for 2 kinds of range specifications when
* creating the Range Request header through the #HTTPClient_AddRangeHeader
* function:
* - When the requested range is all bytes from the starting range byte to
* the end of file.
* - When the requested range is for the last N bytes of the file.
* In both cases, this value should be used for the "rangeEnd" parameter.
*/
#define HTTP_RANGE_REQUEST_END_OF_FILE -1
/**
* @ingroup http_enum_types
* @brief The HTTP Client library return status.
*/
typedef enum HTTPStatus
{
/**
* @brief The HTTP Client library function completed successfully.
*
* Functions that may return this value:
* - #HTTPClient_InitializeRequestHeaders
* - #HTTPClient_AddHeader
* - #HTTPClient_AddRangeHeader
* - #HTTPClient_Send
* - #HTTPClient_ReadHeader
*/
HTTPSuccess,
/**
* @brief The HTTP Client library function input an invalid parameter.
*
* Functions that may return this value:
* - #HTTPClient_InitializeRequestHeaders
* - #HTTPClient_AddHeader
* - #HTTPClient_AddRangeHeader
* - #HTTPClient_Send
* - #HTTPClient_ReadHeader
*/
HTTPInvalidParameter,
/**
* @brief A network error was returned from the transport interface.
*
* Functions that may return this value:
* - #HTTPClient_Send
*/
HTTPNetworkError,
/**
* @brief Part of the HTTP response was received from the network.
*
* Functions that may return this value:
* - #HTTPClient_Send
*/
HTTPPartialResponse,
/**
* @brief No HTTP response was received from the network.
*
* This can occur only if there was no data received from the transport
* interface.
*
* Functions that may return this value:
* - #HTTPClient_Send
*/
HTTPNoResponse,
/**
* @brief The application buffer was not large enough for the HTTP request
* headers or the HTTP response message.
*
* Functions that may return this value:
* - #HTTPClient_InitializeRequestHeaders
* - #HTTPClient_AddHeader
* - #HTTPClient_AddRangeHeader
* - #HTTPClient_Send
*/
HTTPInsufficientMemory,
/**
* @brief The server sent more headers than the configured
* #HTTP_MAX_RESPONSE_HEADERS_SIZE_BYTES.
*
* Functions that may return this value:
* - #HTTPClient_Send
*/
HTTPSecurityAlertResponseHeadersSizeLimitExceeded,
/**
* @brief A response contained the "Connection: close" header, but there
* was more data at the end of the complete message.
*
* Functions that may return this value:
* - #HTTPClient_Send
*/
HTTPSecurityAlertExtraneousResponseData,
/**
* @brief The server sent a chunk header containing an invalid character.
*
* Functions that may return this value:
* - #HTTPClient_Send
*/
HTTPSecurityAlertInvalidChunkHeader,
/**
* @brief The server sent a response with an invalid character in the
* HTTP protocol version.
*
* Functions that may return this value:
* - #HTTPClient_Send
*/
HTTPSecurityAlertInvalidProtocolVersion,
/**
* @brief The server sent a response with an invalid character in the
* HTTP status-code or the HTTP status code is out of range.
*
* Functions that may return this value:
* - #HTTPClient_Send
*/
HTTPSecurityAlertInvalidStatusCode,
/**
* @brief An invalid character was found in the HTTP response message or in
* the HTTP request header.
*
* Functions that may return this value:
* - #HTTPClient_AddHeader
* - #HTTPClient_Send
*/
HTTPSecurityAlertInvalidCharacter,
/**
* @brief The response contains either an invalid character in the
* Content-Length header or a Content-Length header when it was not expected
* to be present.
*
* Functions that may return this value:
* - #HTTPClient_Send
*/
HTTPSecurityAlertInvalidContentLength,
/**
* @brief An error occurred in the third-party parsing library.
*
* Functions that may return this value:
* - #HTTPClient_Send
* - #HTTPClient_ReadHeader
*/
HTTPParserInternalError,
/**
* @brief The requested header field was not found in the response buffer.
*
* Functions that may return this value:
* - #HTTPClient_ReadHeader
*/
HTTPHeaderNotFound,
/**
* @brief The HTTP response, provided for parsing, is either corrupt or
* incomplete.
*
* Functions that may return this value:
* - #HTTPClient_ReadHeader
*/
HTTPInvalidResponse
} HTTPStatus_t;
/**
* @ingroup http_struct_types
* @brief Represents header data that will be sent in an HTTP request.
*
* The memory for the header data buffer is supplied by the user. Information in
* the buffer will be filled by calling #HTTPClient_InitializeRequestHeaders and
* #HTTPClient_AddHeader. This buffer may be automatically filled with the
* Content-Length header in #HTTPClient_Send, please see
* HTTP_MAX_CONTENT_LENGTH_HEADER_LENGTH for the maximum amount of space needed
* to accommodate the Content-Length header.
*/
typedef struct HTTPRequestHeaders
{
/**
* @brief Buffer to hold the raw HTTP request headers.
*
* This buffer is supplied by the application.
*
* This buffer is owned by the library during #HTTPClient_AddHeader,
* #HTTPClient_AddRangeHeader, #HTTPClient_InitializeRequestHeaders, and
* #HTTPClient_Send. This buffer should not be modified until
* after these functions return.
*
* For optimization this buffer may be re-used with the response. The user
* can re-use this buffer for the storing the response from the server in
* #HTTPResponse_t.pBuffer.
*/
uint8_t * pBuffer;
size_t bufferLen; /**< The length of pBuffer in bytes. */
/**
* @brief The actual size in bytes of headers in the buffer. This field
* is updated by the HTTP Client library functions #HTTPClient_AddHeader,
* and #HTTPClient_InitializeRequestHeaders.
*/
size_t headersLen;
} HTTPRequestHeaders_t;
/**
* @ingroup http_struct_types
* @brief Configurations of the initial request headers.
*/
typedef struct HTTPRequestInfo
{
/**
* @brief The HTTP request method e.g. "GET", "POST", "PUT", or "HEAD".
*/
const char * pMethod;
size_t methodLen; /**< The length of the method in bytes. */
/**
* @brief The Request-URI to the objects of interest, e.g. "/path/to/item.txt".
*/
const char * pPath;
size_t pathLen; /**< The length of the path in bytes. */
/**
* @brief The server's host name, e.g. "my-storage.my-cloud.com".
*
* The host does not have a "https://" or "http://" prepending.
*/
const char * pHost;
size_t hostLen; /**< The length of the host in bytes. */
/**
* @brief Flags to activate other request header configurations.
*
* Please see @ref http_request_flags for more information.
*/
uint32_t reqFlags;
} HTTPRequestInfo_t;
/**
* @ingroup http_struct_types
* @brief Callback to intercept headers during the first parse through of the
* response as it is received from the network.
*/
typedef struct HTTPClient_ResponseHeaderParsingCallback
{
/**
* @brief Invoked when both a header field and its associated header value are found.
* @param[in] pContext User context.
* @param[in] fieldLoc Location of the header field name in the response buffer.
* @param[in] fieldLen Length in bytes of the field name.
* @param[in] valueLoc Location of the header value in the response buffer.
* @param[in] valueLen Length in bytes of the value.
* @param[in] statusCode The HTTP response status-code.
*/
void ( * onHeaderCallback )( void * pContext,
const char * fieldLoc,
size_t fieldLen,
const char * valueLoc,
size_t valueLen,
uint16_t statusCode );
/**
* @brief Private context for the application.
*/
void * pContext;
} HTTPClient_ResponseHeaderParsingCallback_t;
/**
* @ingroup http_callback_types
* @brief Application provided function to query the current time in
* milliseconds.
*
* @return The current time in milliseconds.
*/
typedef uint32_t (* HTTPClient_GetCurrentTimeFunc_t )( void );
/**
* @ingroup http_struct_types
* @brief Represents an HTTP response.
*/
typedef struct HTTPResponse
{
/**
* @brief Buffer for both the raw HTTP header and body.
*
* This buffer is supplied by the application.
*
* This buffer is owned by the library during #HTTPClient_Send and
* #HTTPClient_ReadHeader. This buffer should not be modified until after
* these functions return.
*
* For optimization this buffer may be used with the request headers. The
* request header buffer is configured in #HTTPRequestHeaders_t.pBuffer.
* When the same buffer is used for the request headers, #HTTPClient_Send
* will send the headers in the buffer first, then fill the buffer with
* the response message.
*/
uint8_t * pBuffer;
size_t bufferLen; /**< The length of the response buffer in bytes. */
/**
* @brief Optional callback for intercepting the header during the first
* parse through of the response as is it receive from the network.
* Set to NULL to disable.
*/
HTTPClient_ResponseHeaderParsingCallback_t * pHeaderParsingCallback;
/**
* @brief Optional callback for getting the system time.
*
* This is used to calculate the elapsed time when retrying network reads or
* sends that return zero bytes received or sent, respectively. If this
* field is set to NULL, then network send and receive won't be retried
* after a zero is returned.
*
* If this function is set, then the maximum time for retrying network reads
* that return zero bytes can be set through #HTTP_RECV_RETRY_TIMEOUT_MS.
*
* If this function is set, then the maximum elapsed time between network
* sends greater than zero is set in HTTP_SEND_RETRY_TIMEOUT_MS.
*/
HTTPClient_GetCurrentTimeFunc_t getTime;
/**
* @brief The starting location of the response headers in pBuffer.
*
* This is updated by #HTTPClient_Send.
*/
const uint8_t * pHeaders;
/**
* @brief Byte length of the response headers in pBuffer.
*
* This is updated by #HTTPClient_Send.
*/
size_t headersLen;
/**
* @brief The starting location of the response body in pBuffer.
*
* This is updated by #HTTPClient_Send.
*/
const uint8_t * pBody;
/**
* @brief Byte length of the body in pBuffer.
*
* This is updated by #HTTPClient_Send.
*/
size_t bodyLen;
/* Useful HTTP header values found. */
/**
* @brief The HTTP response Status-Code.
*
* This is updated by #HTTPClient_Send.
*/
uint16_t statusCode;
/**
* @brief The value in the "Content-Length" header is returned here.
*
* This is updated by #HTTPClient_Send.
*/
size_t contentLength;
/**
* @brief Count of the headers sent by the server.
*
* This is updated by #HTTPClient_Send.
*/
size_t headerCount;
/**
* @brief Flags of useful headers found in the response.
*
* This is updated by #HTTPClient_Send. Please see @ref http_response_flags
* for more information.
*/
uint32_t respFlags;
int8_t chunked;
int8_t eof;
int8_t has_begin;
uint8_t* chunked_buf;
uint16_t chunked_len;
int8_t eof_time;
} HTTPResponse_t;
/**
* @brief Initialize the request headers, stored in
* #HTTPRequestHeaders_t.pBuffer, with initial configurations from
* #HTTPRequestInfo_t. This method is expected to be called before sending a
* new request.
*
* Upon return, #HTTPRequestHeaders_t.headersLen will be updated with the number
* of bytes written.
*
* Each line in the header is listed below and written in this order:
* <#HTTPRequestInfo_t.pMethod> <#HTTPRequestInfo_t.pPath> <#HTTP_PROTOCOL_VERSION>
* User-Agent: <#HTTP_USER_AGENT_VALUE>
* Host: <#HTTPRequestInfo_t.pHost>
*
* Note that "Connection" header can be added and set to "keep-alive" by
* activating the HTTP_REQUEST_KEEP_ALIVE_FLAG in #HTTPRequestInfo_t.reqFlags.
*
* @param[in] pRequestHeaders Request header buffer information.
* @param[in] pRequestInfo Initial request header configurations.
* @return One of the following:
* - #HTTPSuccess (If successful)
* - #HTTPInvalidParameter (If any provided parameters or their members are invalid.)
* - #HTTPInsufficientMemory (If provided buffer size is not large enough to hold headers.)
*
* **Example**
* @code{c}
* HTTPStatus_t httpLibraryStatus = HTTPSuccess;
* // Declare an HTTPRequestHeaders_t and HTTPRequestInfo_t.
* HTTPRequestHeaders_t requestHeaders = { 0 };
* HTTPRequestInfo_t requestInfo = { 0 };
* // A buffer that will fit the Request-Line, the User-Agent header line, and
* // the Host header line.
* uint8_t requestHeaderBuffer[ 256 ] = { 0 };
*
* // Set a buffer to serialize request headers to.
* requestHeaders.pBuffer = requestHeaderBuffer;
* requestHeaders.bufferLen = 256;
*
* // Set the Method, Path, and Host in the HTTPRequestInfo_t.
* requestInfo.pMethod = HTTP_METHOD_GET;
* requestInfo.methodLen = sizeof( HTTP_METHOD_GET ) - 1U;
* requestInfo.pPath = "/html/rfc2616"
* requestInfo.pathLen = sizeof( "/html/rfc2616" ) - 1U;
* requestInfo.pHost = "tools.ietf.org"
* requestInfo.hostLen = sizeof( "tools.ietf.org" ) - 1U;
* requestInfo.reqFlags |= HTTP_REQUEST_KEEP_ALIVE_FLAG;
*
* httpLibraryStatus = HTTPClient_InitializeRequestHeaders( &requestHeaders,
* &requestInfo );
* @endcode
*/
/* @[declare_httpclient_initializerequestheaders] */
HTTPStatus_t HTTPClient_InitializeRequestHeaders( HTTPRequestHeaders_t * pRequestHeaders,
const HTTPRequestInfo_t * pRequestInfo );
/* @[declare_httpclient_initializerequestheaders] */
/**
* @brief Add a header to the request headers stored in
* #HTTPRequestHeaders_t.pBuffer.
*
* Upon return, pRequestHeaders->headersLen will be updated with the number of
* bytes written.
*
* Headers are written in the following format:
*
* @code
* <field>: <value>\r\n\r\n
* @endcode
*
* The trailing `\r\n` that denotes the end of the header lines is overwritten,
* if it already exists in the buffer.
*
* @note This function validates only that `\r`, `\n`, and `:` are not present
* in @p pValue or @p pField. `:` is allowed in @p pValue.
*
* @param[in] pRequestHeaders Request header buffer information.
* @param[in] pField The header field name to write.
* The data should be ISO 8859-1 (Latin-1) encoded per the HTTP standard,
* but the API does not perform the character set validation.
* @param[in] fieldLen The byte length of the header field name.
* @param[in] pValue The header value to write.
* The data should be ISO 8859-1 (Latin-1) encoded per the HTTP standard,
* but the API does not perform the character set validation.
* @param[in] valueLen The byte length of the header field value.
*
* @return One of the following:
* - #HTTPSuccess (If successful.)
* - #HTTPInvalidParameter (If any provided parameters or their members are invalid.)
* - #HTTPInsufficientMemory (If application-provided buffer is not large enough to hold headers.)
* - #HTTPSecurityAlertInvalidCharacter (If an invalid character was found in @p pField or @p pValue.)
*
* **Example**
* @code{c}
* HTTPStatus_t httpLibraryStatus = HTTPSuccess;
* // Assume that requestHeaders has already been initialized with
* // HTTPClient_InitializeRequestHeaders().
* HTTPRequestHeaders_t requestHeaders;
*
* httpLibraryStatus = HTTPClient_AddHeader( &requestHeaders,
* "Request-Header-Field",
* sizeof( "Request-Header-Field" ) - 1U,
* "Request-Header-Value",
* sizeof("Request-Header-Value") - 1U );
* @endcode
*/
/* @[declare_httpclient_addheader] */
HTTPStatus_t HTTPClient_AddHeader( HTTPRequestHeaders_t * pRequestHeaders,
const char * pField,
size_t fieldLen,
const char * pValue,
size_t valueLen );
/* @[declare_httpclient_addheader] */
/**
* @brief Add the byte range request header to the request headers store in
* #HTTPRequestHeaders_t.pBuffer.
*
* For example, if requesting for the first 1kB of a file the following would be
* written: `Range: bytes=0-1023\r\n\r\n`.
*
* The trailing `\r\n` that denotes the end of the header lines is overwritten,
* if it already exists in the buffer.
*
* There are 3 different forms of range specification, determined by the
* combination of @a rangeStartOrLastNBytes and @a rangeEnd parameter values:
*
* 1. Request containing both parameters for the byte range [rangeStart, rangeEnd]
* where @a rangeStartOrLastNBytes <= @a rangeEnd.
* Example request header line: `Range: bytes=0-1023\r\n` for requesting bytes in the range [0, 1023].<br>
* **Example**
* @code{c}
* HTTPStatus_t httpLibraryStatus = HTTPSuccess;
* // Assume that requestHeaders has already been initialized with
* // HTTPClient_InitializeRequestHeaders().
* HTTPRequestHeaders_t requestHeaders;
*
* // Request for bytes 0 to 1023.
* httpLibraryStatus = HTTPClient_AddRangeHeader( &requestHeaders, 0, 1023 );
* @endcode
*
* 2. Request for the last N bytes, represented by @p rangeStartOrlastNbytes.
* @p rangeStartOrlastNbytes should be negative and @p rangeEnd should be
* #HTTP_RANGE_REQUEST_END_OF_FILE.
* Example request header line: `Range: bytes=-512\r\n` for requesting the last 512 bytes
* (or bytes in the range [512, 1023] for a 1KB sized file).<br>
* **Example**
* @code{c}
* HTTPStatus_t httpLibraryStatus = HTTPSuccess;
* // Assume that requestHeaders has already been initialized with
* // HTTPClient_InitializeRequestHeaders().
* HTTPRequestHeaders_t requestHeaders;
*
* // Request for the last 512 bytes.
* httpLibraryStatus = HTTPClient_AddRangeHeader( &requestHeaders, -512, HTTP_RANGE_REQUEST_END_OF_FILE)
* @endcode
*
* 3. Request for all bytes (till the end of byte sequence) from byte N,
* represented by @p rangeStartOrlastNbytes.
* @p rangeStartOrlastNbytes should be >= 0 and @p rangeEnd should be
* #HTTP_RANGE_REQUEST_END_OF_FILE.<br>
* Example request header line: `Range: bytes=256-\r\n` for requesting all bytes after and
* including byte 256 (or bytes in the range [256,1023] for a 1kB sized file).<br>
* **Example**
* @code{c}
* HTTPStatus_t httpLibraryStatus = HTTPSuccess;
* // Assume that requestHeaders has already been initialized with
* // HTTPClient_InitializeRequestHeaders().
* HTTPRequestHeaders_t requestHeaders;
*
* // Request for all bytes from byte 256 onward.
* httpLibraryStatus = HTTPClient_AddRangeHeader( &requestHeaders, 256, HTTP_RANGE_REQUEST_END_OF_FILE)
* @endcode
*
* @param[in] pRequestHeaders Request header buffer information.
* @param[in] rangeStartOrlastNbytes Represents either the starting byte
* for a range OR the last N number of bytes in the requested file.
* @param[in] rangeEnd The ending range for the requested file. For end of file
* byte in Range Specifications 2. and 3., #HTTP_RANGE_REQUEST_END_OF_FILE
* should be passed.
*
* @return Returns the following status codes:
* #HTTPSuccess, if successful.
* #HTTPInvalidParameter, if input parameters are invalid, including when
* the @p rangeStartOrlastNbytes and @p rangeEnd parameter combination is invalid.
* #HTTPInsufficientMemory, if the passed #HTTPRequestHeaders_t.pBuffer
* contains insufficient remaining memory for storing the range request.
*/
/* @[declare_httpclient_addrangeheader] */
HTTPStatus_t HTTPClient_AddRangeHeader( HTTPRequestHeaders_t * pRequestHeaders,
int32_t rangeStartOrlastNbytes,
int32_t rangeEnd );
/* @[declare_httpclient_addrangeheader] */
/**
* @brief Send the request headers in #HTTPRequestHeaders_t.pBuffer and request
* body in @p pRequestBodyBuf over the transport. The response is received in
* #HTTPResponse_t.pBuffer.
*
* If #HTTP_SEND_DISABLE_CONTENT_LENGTH_FLAG is not set in parameter @p sendFlags,
* then the Content-Length to be sent to the server is automatically written to
* @p pRequestHeaders. The Content-Length will not be written when there is
* no request body. If there is not enough room in the buffer to write the
* Content-Length then #HTTPInsufficientMemory is returned. Please see
* #HTTP_MAX_CONTENT_LENGTH_HEADER_LENGTH for the maximum Content-Length header
* field and value that could be written to the buffer.
*
* The application should close the connection with the server if any of the
* following errors are returned:
* - #HTTPSecurityAlertResponseHeadersSizeLimitExceeded
* - #HTTPSecurityAlertExtraneousResponseData
* - #HTTPSecurityAlertInvalidChunkHeader
* - #HTTPSecurityAlertInvalidProtocolVersion
* - #HTTPSecurityAlertInvalidStatusCode
* - #HTTPSecurityAlertInvalidCharacter
* - #HTTPSecurityAlertInvalidContentLength
*
* The @p pResponse returned is valid only if this function returns HTTPSuccess.
*
* @param[in] pTransport Transport interface, see #TransportInterface_t for
* more information.
* @param[in] pRequestHeaders Request configuration containing the buffer of
* headers to send.
* @param[in] pRequestBodyBuf Optional Request entity body. Set to NULL if there
* is no request body.
* @param[in] reqBodyBufLen The length of the request entity in bytes.
* @param[in] pResponse The response message and some notable response
* parameters will be returned here on success.
* @param[in] sendFlags Flags which modify the behavior of this function. Please
* see @ref http_send_flags for more information.
*
* @return One of the following:
* - #HTTPSuccess (If successful.)
* - #HTTPInvalidParameter (If any provided parameters or their members are invalid.)
* - #HTTPNetworkError (Errors in sending or receiving over the transport interface.)
* - #HTTPPartialResponse (Part of an HTTP response was received in a partially filled response buffer.)
* - #HTTPNoResponse (No data was received from the transport interface.)
* - #HTTPInsufficientMemory (The response received could not fit into the response buffer
* or extra headers could not be sent in the request.)
* - #HTTPParserInternalError (Internal parsing error.)\n\n
* Security alerts are listed below, please see #HTTPStatus_t for more information:
* - #HTTPSecurityAlertResponseHeadersSizeLimitExceeded
* - #HTTPSecurityAlertExtraneousResponseData
* - #HTTPSecurityAlertInvalidChunkHeader
* - #HTTPSecurityAlertInvalidProtocolVersion
* - #HTTPSecurityAlertInvalidStatusCode
* - #HTTPSecurityAlertInvalidCharacter
* - #HTTPSecurityAlertInvalidContentLength
*
* **Example**
* @code{c}
* // Variables used in this example.
* HTTPStatus_t httpLibraryStatus = HTTPSuccess;
* TransportInterface_t transportInterface = { 0 };
* HTTPResponse_t = { 0 };
* char requestBody[] = "This is an example request body.";
*
* // Assume that requestHeaders has been initialized with
* // HTTPClient_InitializeResponseHeaders() and any additional headers have
* // been added with HTTPClient_AddHeader().
* HTTPRequestHeaders_t requestHeaders;
*
* // Set the transport interface with platform specific functions that are
* // assumed to be implemented elsewhere.
* transportInterface.recv = myPlatformTransportReceive;
* transportInterface.send = myPlatformTransportSend;
* transportInterface.pNetworkContext = myPlatformNetworkContext;
*
* // Set the buffer to receive the HTTP response message into. The buffer is
* // dynamically allocated for demonstration purposes only.
* response.pBuffer = ( uint8_t* )malloc( 1024 );
* response.bufferLen = 1024;
*
* httpLibraryStatus = HTTPClient_Send( &transportInterface,
* &requestHeaders,
* requestBody,
* sizeof( requestBody ) - 1U,
* &response,
* 0 );
*
* if( httpLibraryStatus == HTTPSuccess )
* {
* if( response.status == 200 )
* {
* // Handle a response Status-Code of 200 OK.
* }
* }
* @endcode
*/
/* @[declare_httpclient_send] */
HTTPStatus_t HTTPClient_Send( const TransportInterface_t * pTransport,
HTTPRequestHeaders_t * pRequestHeaders,
const uint8_t * pRequestBodyBuf,
size_t reqBodyBufLen,
HTTPResponse_t * pResponse,
uint32_t sendFlags );
/* @[declare_httpclient_send] */
HTTPStatus_t HTTPClient_Recv(TransportInterface_t *pTransport, HTTPResponse_t *pResponse);
HTTPStatus_t HTTPClient_Seek(
const TransportInterface_t *pTransport, HTTPRequestHeaders_t *pRequestHeaders);
/**
* @brief Read a header from a buffer containing a complete HTTP response.
* This will return the location of the response header value in the
* #HTTPResponse_t.pBuffer buffer.
*
* The location, within #HTTPResponse_t.pBuffer, of the value found, will be
* returned in @p pValue. If the header value is empty for the found @p pField,
* then this function will return #HTTPSuccess, and set the values for
* @p pValueLoc and @p pValueLen as NULL and zero respectively. According to
* RFC 2616, it is not invalid to have an empty value for some header fields.
*
* @note This function should only be called on a complete HTTP response. If the
* request is sent through the #HTTPClient_Send function, the #HTTPResponse_t is
* incomplete until #HTTPClient_Send returns.
*
* @param[in] pResponse The buffer containing the completed HTTP response.
* @param[in] pField The header field name to read.
* @param[in] fieldLen The length of the header field name in bytes.
* @param[out] pValueLoc This will be populated with the location of the
* header value in the response buffer, #HTTPResponse_t.pBuffer.
* @param[out] pValueLen This will be populated with the length of the
* header value in bytes.
*
* @return One of the following:
* - #HTTPSuccess (If successful.)
* - #HTTPInvalidParameter (If any provided parameters or their members are invalid.)
* - #HTTPHeaderNotFound (Header is not found in the passed response buffer.)
* - #HTTPInvalidResponse (Provided response is not a valid HTTP response for parsing.)
* - #HTTPParserInternalError(If an error in the response parser.)
*
* **Example**
* @code{c}
* HTTPStatus_t httpLibraryStatus = HTTPSuccess;
* // Assume that response is returned from a successful invocation of
* // HTTPClient_Send().
* HTTPResponse_t response;
*
* char * pDateLoc = NULL;
* size_t dateLen = 0;
* // Search for a "Date" header field. pDateLoc will be the location of the
* // Date header value in response.pBuffer.
* httpLibraryStatus = HTTPClient_ReadHeader( &response,
* "Date",
* sizeof("Date") - 1,
* &pDateLoc,
* &dateLen );
* @endcode
*/
/* @[declare_httpclient_readheader] */
HTTPStatus_t HTTPClient_ReadHeader( const HTTPResponse_t * pResponse,
const char * pField,
size_t fieldLen,
const char ** pValueLoc,
size_t * pValueLen );
/* @[declare_httpclient_readheader] */
/**
* @brief Error code to string conversion utility for HTTP Client library.
*
* @note This returns constant strings, which should not be modified.
*
* @param[in] status The status code to convert to a string.
*
* @return The string representation of the status code.
*/
/* @[declare_httpclient_strerror] */
const char * HTTPClient_strerror( HTTPStatus_t status );
/* @[declare_httpclient_strerror] */
/* *INDENT-OFF* */
#ifdef __cplusplus
}
#endif
/* *INDENT-ON* */
#endif /* ifndef CORE_HTTP_CLIENT_H_ */

282
arcs-sdk/modules/coreHTTP/source/include/core_http_client_private.h Normal file
View File

@@ -0,0 +1,282 @@
/*
* coreHTTP v2.1.0
* Copyright (C) 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of
* this software and associated documentation files (the "Software"), to deal in
* the Software without restriction, including without limitation the rights to
* use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
* the Software, and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
* IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
/**
* @file core_http_client_private.h
* @brief Internal definitions to the HTTP Client library.
*/
#ifndef CORE_HTTP_CLIENT_PRIVATE_H_
#define CORE_HTTP_CLIENT_PRIVATE_H_
/* Third-party http-parser include. */
#include "http_parser.h"
/* *INDENT-OFF* */
#ifdef __cplusplus
extern "C" {
#endif
/* *INDENT-ON* */
/**
* @brief The HTTP protocol version of this library is HTTP/1.1.
*/
#define HTTP_PROTOCOL_VERSION "HTTP/1.1"
#define HTTP_PROTOCOL_VERSION_LEN ( sizeof( HTTP_PROTOCOL_VERSION ) - 1U ) /**< The length of #HTTP_PROTOCOL_VERSION. */
/**
* @brief Default value when pRequestInfo->pPath == NULL.
*/
#define HTTP_EMPTY_PATH "/"
#define HTTP_EMPTY_PATH_LEN ( sizeof( HTTP_EMPTY_PATH ) - 1U ) /**< The length of #HTTP_EMPTY_PATH. */
/* Constants for HTTP header formatting. */
#define HTTP_HEADER_LINE_SEPARATOR "\r\n" /**< HTTP header field lines are separated by `\r\n`. */
#define HTTP_HEADER_LINE_SEPARATOR_LEN ( sizeof( HTTP_HEADER_LINE_SEPARATOR ) - 1U ) /**< The length of #HTTP_HEADER_LINE_SEPARATOR. */
#define HTTP_HEADER_END_INDICATOR "\r\n\r\n" /**< The HTTP header is complete when `\r\n\r\n` is found. */
#define HTTP_HEADER_END_INDICATOR_LEN ( sizeof( HTTP_HEADER_END_INDICATOR ) - 1U ) /**< The length of #HTTP_HEADER_END_INDICATOR. */
#define HTTP_HEADER_FIELD_SEPARATOR ": " /**< HTTP header field and values are separated by ": ". */
#define HTTP_HEADER_FIELD_SEPARATOR_LEN ( sizeof( HTTP_HEADER_FIELD_SEPARATOR ) - 1U ) /**< The length of #HTTP_HEADER_FIELD_SEPARATOR. */
#define SPACE_CHARACTER ' ' /**< A space character macro to help with serializing a request. */
#define SPACE_CHARACTER_LEN ( 1U ) /**< The length of #SPACE_CHARACTER. */
#define DASH_CHARACTER '-' /**< A dash character macro to help with serializing a request. */
#define DASH_CHARACTER_LEN ( 1U ) /**< The length of #DASH_CHARACTER. */
/* Constants for HTTP header copy checks. */
#define CARRIAGE_RETURN_CHARACTER '\r' /**< A carriage return character to help with header validation. */
#define LINEFEED_CHARACTER '\n' /**< A linefeed character to help with header validation. */
#define COLON_CHARACTER ':' /**< A colon character to help with header validation. */
/**
* @brief Indicator for function #httpHeaderStrncpy that the pSrc parameter is a
* header value.
*/
#define HTTP_HEADER_STRNCPY_IS_VALUE 0U
/**
* @brief Indicator for function #httpHeaderStrncpy that the pSrc parameter is a
* header field.
*/
#define HTTP_HEADER_STRNCPY_IS_FIELD 1U
/* Constants for header fields added automatically during the request
* initialization. */
#define HTTP_USER_AGENT_FIELD "User-Agent" /**< HTTP header field "User-Agent". */
#define HTTP_USER_AGENT_FIELD_LEN ( sizeof( HTTP_USER_AGENT_FIELD ) - 1U ) /**< The length of #HTTP_USER_AGENT_FIELD. */
#define HTTP_HOST_FIELD "Host" /**< HTTP header field "Host". */
#define HTTP_HOST_FIELD_LEN ( sizeof( HTTP_HOST_FIELD ) - 1U ) /**< The length of #HTTP_HOST_FIELD. */
#define HTTP_USER_AGENT_VALUE_LEN ( sizeof( HTTP_USER_AGENT_VALUE ) - 1U ) /**< The length of #HTTP_USER_AGENT_VALUE. */
/* Constants for header fields added based on flags. */
#define HTTP_CONNECTION_FIELD "Connection" /**< HTTP header field "Connection". */
#define HTTP_CONNECTION_FIELD_LEN ( sizeof( HTTP_CONNECTION_FIELD ) - 1U ) /**< The length of #HTTP_CONNECTION_FIELD. */
#define HTTP_CONTENT_LENGTH_FIELD "Content-Length" /**< HTTP header field "Content-Length". */
#define HTTP_CONTENT_LENGTH_FIELD_LEN ( sizeof( HTTP_CONTENT_LENGTH_FIELD ) - 1U ) /**< The length of #HTTP_CONTENT_LENGTH_FIELD. */
/* Constants for header values added based on flags. */
/* MISRA Rule 5.4 flags the following macro's name as ambiguous from the
* one postfixed with _LEN. This rule is suppressed for naming consistency with
* other HTTP header field and value string and length macros in this file.*/
/* coverity[other_declaration] */
#define HTTP_CONNECTION_KEEP_ALIVE_VALUE "keep-alive" /**< HTTP header value "keep-alive" for the "Connection" header field. */
/* MISRA Rule 5.4 flags the following macro's name as ambiguous from the one
* above it. This rule is suppressed for naming consistency with other HTTP
* header field and value string and length macros in this file.*/
/* coverity[misra_c_2012_rule_5_4_violation] */
#define HTTP_CONNECTION_KEEP_ALIVE_VALUE_LEN ( sizeof( HTTP_CONNECTION_KEEP_ALIVE_VALUE ) - 1U ) /**< The length of #HTTP_CONNECTION_KEEP_ALIVE_VALUE. */
/* Constants relating to Range Requests. */
/* MISRA Rule 5.4 flags the following macro's name as ambiguous from the
* one postfixed with _LEN. This rule is suppressed for naming consistency with
* other HTTP header field and value string and length macros in this file.*/
/* coverity[other_declaration] */
#define HTTP_RANGE_REQUEST_HEADER_FIELD "Range" /**< HTTP header field "Range". */
/* MISRA Rule 5.4 flags the following macro's name as ambiguous from the one
* above it. This rule is suppressed for naming consistency with other HTTP
* header field and value string and length macros in this file.*/
/* coverity[misra_c_2012_rule_5_4_violation] */
#define HTTP_RANGE_REQUEST_HEADER_FIELD_LEN ( sizeof( HTTP_RANGE_REQUEST_HEADER_FIELD ) - 1U ) /**< The length of #HTTP_RANGE_REQUEST_HEADER_FIELD. */
/* MISRA Rule 5.4 flags the following macro's name as ambiguous from the
* one postfixed with _LEN. This rule is suppressed for naming consistency with
* other HTTP header field and value string and length macros in this file.*/
/* coverity[other_declaration] */
#define HTTP_RANGE_REQUEST_HEADER_VALUE_PREFIX "bytes=" /**< HTTP required header value prefix when specifying a byte range for partial content. */
/* MISRA Rule 5.4 flags the following macro's name as ambiguous from the one
* above it. This rule is suppressed for naming consistency with other HTTP
* header field and value string and length macros in this file.*/
/* coverity[misra_c_2012_rule_5_4_violation] */
#define HTTP_RANGE_REQUEST_HEADER_VALUE_PREFIX_LEN ( sizeof( HTTP_RANGE_REQUEST_HEADER_VALUE_PREFIX ) - 1U ) /**< The length of #HTTP_RANGE_REQUEST_HEADER_VALUE_PREFIX. */
/**
* @brief Maximum value of a 32 bit signed integer is 2,147,483,647.
*
* Used for calculating buffer space for ASCII representation of range values.
*/
#define MAX_INT32_NO_OF_DECIMAL_DIGITS 10U
/**
* @brief Maximum buffer space for storing a Range Request Value.
*
* The largest Range Request value is of the form:
* "bytes=<Max-Integer-Value>-<Max-Integer-Value>"
*/
#define HTTP_MAX_RANGE_REQUEST_VALUE_LEN \
( HTTP_RANGE_REQUEST_HEADER_VALUE_PREFIX_LEN + MAX_INT32_NO_OF_DECIMAL_DIGITS + \
1U /* Dash character '-' */ + MAX_INT32_NO_OF_DECIMAL_DIGITS )
/**
* @brief Return value for the http-parser registered callback to signal halting
* further execution.
*/
#define HTTP_PARSER_STOP_PARSING 1
/**
* @brief Return value for http_parser registered callback to signal
* continuation of HTTP response parsing.
*/
#define HTTP_PARSER_CONTINUE_PARSING 0
/**
* @brief The minimum request-line in the headers has a possible one character
* custom method and a single forward / or asterisk * for the path:
*
* @code
* <1 character custom method> <1 character / or *> HTTP/1.x\r\n\r\n
* @endcode
*
* Therefore the minimum length is 16. If this minimum request-line is not
* satisfied, then the request headers to send are invalid.
*
* Note that custom methods are allowed per:
* https://tools.ietf.org/html/rfc2616#section-5.1.1.
*/
#define HTTP_MINIMUM_REQUEST_LINE_LENGTH 16u
/**
* @brief The state of the response message parsed after function
* #parseHttpResponse returns.
*/
typedef enum HTTPParsingState_t
{
HTTP_PARSING_NONE = 0, /**< The parser has not started reading any response. */
HTTP_PARSING_INCOMPLETE, /**< The parser found a partial reponse. */
HTTP_PARSING_COMPLETE /**< The parser found the entire response. */
} HTTPParsingState_t;
/**
* @brief An aggregator that represents the user-provided parameters to the
* #HTTPClient_ReadHeader API function. This will be used as context parameter
* for the parsing callbacks used by the API function.
*/
typedef struct findHeaderContext
{
const char * pField; /**< The field that is being searched for. */
size_t fieldLen; /**< The length of pField. */
const char ** pValueLoc; /**< The location of the value found in the buffer. */
size_t * pValueLen; /**< the length of the value found. */
uint8_t fieldFound; /**< Indicates that the header field was found during parsing. */
uint8_t valueFound; /**< Indicates that the header value was found during parsing. */
} findHeaderContext_t;
/**
* @brief The HTTP response parsing context for a response fresh from the
* server. This context is passed into the http-parser registered callbacks.
* The registered callbacks are private functions of the form
* httpParserXXXXCallbacks().
*
* The transitions of the httpParserXXXXCallback() functions are shown below.
* The XXXX is replaced by the strings in the state boxes:
*
* +---------------------+
* |onMessageBegin |
* +--------+------------+
* |
* |
* |
* v
* +--------+------------+
* |onStatus |
* +--------+------------+
* |
* |
* |
* v
* +--------+------------+
* |onHeaderField +<---+
* +--------+------------+ |
* | |
* | |(More headers)
* | |
* v |
* +--------+------------+ |
* |onHeaderValue +----^
* +--------+------------+
* |
* |
* |
* v
* +--------+------------+
* |onHeadersComplete |
* +---------------------+
* |
* |
* |
* v
* +--------+------------+
* |onBody +<---+
* +--------+--------+---+ |
* | | |(Transfer-encoding chunked body)
* | | |
* | +--------+
* |
* v
* +--------+------------+
* |onMessageComplete |
* +---------------------+
*/
typedef struct HTTPParsingContext
{
http_parser httpParser; /**< Third-party http-parser context. */
HTTPParsingState_t state; /**< The current state of the HTTP response parsed. */
struct HTTPResponse * pResponse; /**< HTTP response associated with this parsing context. */
uint8_t isHeadResponse; /**< HTTP response is for a HEAD request. */
const char * pBufferCur; /**< The current location of the parser in the response buffer. */
const char * pLastHeaderField; /**< Holds the last part of the header field parsed. */
size_t lastHeaderFieldLen; /**< The length of the last header field parsed. */
const char * pLastHeaderValue; /**< Holds the last part of the header value parsed. */
size_t lastHeaderValueLen; /**< The length of the last value field parsed. */
} HTTPParsingContext_t;
/* *INDENT-OFF* */
#ifdef __cplusplus
}
#endif
/* *INDENT-ON* */
#endif /* ifndef CORE_HTTP_CLIENT_PRIVATE_H_ */

206
arcs-sdk/modules/coreHTTP/source/include/core_http_config_defaults.h Normal file
View File

@@ -0,0 +1,206 @@
/*
* coreHTTP v2.1.0
* Copyright (C) 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of
* this software and associated documentation files (the "Software"), to deal in
* the Software without restriction, including without limitation the rights to
* use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
* the Software, and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
* IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
/**
* @file core_http_config_defaults.h
* @brief The default values for the configuration macros for the HTTP Client
* library.
*
* @note This file SHOULD NOT be modified. If custom values are needed for
* any configuration macro, a core_http_config.h file should be provided to
* the HTTP Client library to override the default values defined in this file.
* To use the custom config file, the HTTP_DO_NOT_USE_CUSTOM_CONFIG preprocessor
* macro SHOULD NOT be set.
*/
#ifndef CORE_HTTP_CONFIG_DEFAULTS_
#define CORE_HTTP_CONFIG_DEFAULTS_
/* *INDENT-OFF* */
#ifdef __cplusplus
extern "C" {
#endif
/* *INDENT-ON* */
/**
* @brief Maximum size, in bytes, of headers allowed from the server.
*
* If the total size in bytes of the headers received from the server exceeds
* this configuration, then the status code
* #HTTPSecurityAlertResponseHeadersSizeLimitExceeded is returned from
* #HTTPClient_Send.
*
* <b>Possible values:</b> Any positive 32 bit integer. <br>
* <b>Default value:</b> `2048`
*/
#ifndef HTTP_MAX_RESPONSE_HEADERS_SIZE_BYTES
#define HTTP_MAX_RESPONSE_HEADERS_SIZE_BYTES 2048U
#endif
/**
* @brief The HTTP header "User-Agent" value.
*
* The following header line is automatically written to
* #HTTPRequestHeaders_t.pBuffer:
* "User-Agent: my-platform-name\r\n"
*
* <b>Possible values:</b> Any string. <br>
* <b>Default value:</b> `my-platform-name`
*/
#ifndef HTTP_USER_AGENT_VALUE
#define HTTP_USER_AGENT_VALUE "Mozilla/5.0"
#endif
/**
* @brief The maximum duration between non-empty network reads while receiving
* an HTTP response via the #HTTPClient_Send API function.
*
* The transport receive function may be called multiple times until the end of
* the response is detected by the parser. This timeout represents the maximum
* duration that is allowed without any data reception from the network for the
* incoming response.
*
* If the timeout expires, the #HTTPClient_Send function will return
* #HTTPNetworkError.
*
* If #HTTPResponse_t.getTime is set to NULL, then this HTTP_RECV_RETRY_TIMEOUT_MS
* is unused. When this timeout is unused, #HTTPClient_Send will not retry the
* transport receive calls that return zero bytes read.
*
* <b>Possible values:</b> Any positive 32 bit integer. A small timeout value
* is recommended. <br>
* <b>Default value:</b> `10`
*/
#ifndef HTTP_RECV_RETRY_TIMEOUT_MS
#define HTTP_RECV_RETRY_TIMEOUT_MS ( 10U )
#endif
/**
* @brief The maximum duration between non-empty network transmissions while
* sending an HTTP request via the #HTTPClient_Send API function.
*
* When sending an HTTP request, the transport send function may be called multiple
* times until all of the required number of bytes are sent.
* This timeout represents the maximum duration that is allowed for no data
* transmission over the network through the transport send function.
*
* If the timeout expires, the #HTTPClient_Send function returns #HTTPNetworkError.
*
* If #HTTPResponse_t.getTime is set to NULL, then this HTTP_RECV_RETRY_TIMEOUT_MS
* is unused. When this timeout is unused, #HTTPClient_Send will not retry the
* transport send calls that return zero bytes sent.
*
* <b>Possible values:</b> Any positive 32 bit integer. A small timeout value
* is recommended. <br>
* <b>Default value:</b> `10`
*/
#ifndef HTTP_SEND_RETRY_TIMEOUT_MS
#define HTTP_SEND_RETRY_TIMEOUT_MS ( 10U )
#endif
/**
* @brief Macro that is called in the HTTP Client library for logging "Error" level
* messages.
*
* To enable error level logging in the HTTP Client library, this macro should be mapped to the
* application-specific logging implementation that supports error logging.
*
* @note This logging macro is called in the HTTP Client library with parameters wrapped in
* double parentheses to be ISO C89/C90 standard compliant. For a reference
* POSIX implementation of the logging macros, refer to core_http_config.h files, and the
* logging-stack in demos folder of the
* [AWS IoT Embedded C SDK repository](https://github.com/aws/aws-iot-device-sdk-embedded-C).
*
* <b>Default value</b>: Error logging is turned off, and no code is generated for calls
* to the macro in the HTTP Client library on compilation.
*/
#ifndef LogError
#define LogError( message )
#endif
/**
* @brief Macro that is called in the HTTP Client library for logging "Warning" level
* messages.
*
* To enable warning level logging in the HTTP Client library, this macro should be mapped to the
* application-specific logging implementation that supports warning logging.
*
* @note This logging macro is called in the HTTP Client library with parameters wrapped in
* double parentheses to be ISO C89/C90 standard compliant. For a reference
* POSIX implementation of the logging macros, refer to core_http_config.h files, and the
* logging-stack in demos folder of the
* [AWS IoT Embedded C SDK repository](https://github.com/aws/aws-iot-device-sdk-embedded-C).
*
* <b>Default value</b>: Warning logs are turned off, and no code is generated for calls
* to the macro in the HTTP Client library on compilation.
*/
#ifndef LogWarn
#define LogWarn( message )
#endif
/**
* @brief Macro that is called in the HTTP Client library for logging "Info" level
* messages.
*
* To enable info level logging in the HTTP Client library, this macro should be mapped to the
* application-specific logging implementation that supports info logging.
*
* @note This logging macro is called in the HTTP Client library with parameters wrapped in
* double parentheses to be ISO C89/C90 standard compliant. For a reference
* POSIX implementation of the logging macros, refer to core_http_config.h files, and the
* logging-stack in demos folder of the
* [AWS IoT Embedded C SDK repository](https://github.com/aws/aws-iot-device-sdk-embedded-C).
*
* <b>Default value</b>: Info logging is turned off, and no code is generated for calls
* to the macro in the HTTP Client library on compilation.
*/
#ifndef LogInfo
#define LogInfo( message )
#endif
/**
* @brief Macro that is called in the HTTP Client library for logging "Debug" level
* messages.
*
* To enable debug level logging from HTTP Client library, this macro should be mapped to the
* application-specific logging implementation that supports debug logging.
*
* @note This logging macro is called in the HTTP Client library with parameters wrapped in
* double parentheses to be ISO C89/C90 standard compliant. For a reference
* POSIX implementation of the logging macros, refer to core_http_config.h files, and the
* logging-stack in demos folder of the
* [AWS IoT Embedded C SDK repository](https://github.com/aws/aws-iot-device-sdk-embedded-C).
*
* <b>Default value</b>: Debug logging is turned off, and no code is generated for calls
* to the macro in the HTTP Client library on compilation.
*/
#ifndef LogDebug
#define LogDebug( message )
#endif
/* *INDENT-OFF* */
#ifdef __cplusplus
}
#endif
/* *INDENT-ON* */
#endif /* ifndef CORE_HTTP_CONFIG_DEFAULTS_ */

267
arcs-sdk/modules/coreHTTP/source/interface/transport_interface.h Normal file
View File

@@ -0,0 +1,267 @@
/*
* coreHTTP v2.1.0
* Copyright (C) 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of
* this software and associated documentation files (the "Software"), to deal in
* the Software without restriction, including without limitation the rights to
* use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
* the Software, and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
* IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
/**
* @file transport_interface.h
* @brief Transport interface definitions to send and receive data over the
* network.
*/
#ifndef TRANSPORT_INTERFACE_H_
#define TRANSPORT_INTERFACE_H_
#include <stdint.h>
#include <stddef.h>
#include "http_parser.h"
/* *INDENT-OFF* */
#ifdef __cplusplus
extern "C" {
#endif
/* *INDENT-ON* */
/**
* @transportpage
* @brief The transport interface definition.
*
* @transportsectionoverview
*
* The transport interface is a set of APIs that must be implemented using an
* external transport layer protocol. The transport interface is defined in
* @ref transport_interface.h. This interface allows protocols like MQTT and
* HTTP to send and receive data over the transport layer. This
* interface does not handle connection and disconnection to the server of
* interest. The connection, disconnection, and other transport settings, like
* timeout and TLS setup, must be handled in the user application.
* <br>
*
* The functions that must be implemented are:<br>
* - [Transport Receive](@ref TransportRecv_t)
* - [Transport Send](@ref TransportSend_t)
*
* Each of the functions above take in an opaque context @ref NetworkContext_t.
* The functions above and the context are also grouped together in the
* @ref TransportInterface_t structure:<br><br>
* @snippet this define_transportinterface
* <br>
*
* @transportsectionimplementation
*
* The following steps give guidance on implementing the transport interface:
*
* -# Implementing @ref NetworkContext_t<br><br>
* @snippet this define_networkcontext
* <br>
* @ref NetworkContext_t is the incomplete type <b>struct NetworkContext</b>.
* The implemented struct NetworkContext must contain all of the information
* that is needed to receive and send data with the @ref TransportRecv_t
* and the @ref TransportSend_t implementations.<br>
* In the case of TLS over TCP, struct NetworkContext is typically implemented
* with the TCP socket context and a TLS context.<br><br>
* <b>Example code:</b>
* @code{c}
* struct NetworkContext
* {
* struct MyTCPSocketContext tcpSocketContext;
* struct MyTLSContext tlsContext;
* };
* @endcode
* <br>
* -# Implementing @ref TransportRecv_t<br><br>
* @snippet this define_transportrecv
* <br>
* This function is expected to populate a buffer, with bytes received from the
* transport, and return the number of bytes placed in the buffer.
* In the case of TLS over TCP, @ref TransportRecv_t is typically implemented by
* calling the TLS layer function to receive data. In case of plaintext TCP
* without TLS, it is typically implemented by calling the TCP layer receive
* function. @ref TransportRecv_t may be invoked multiple times by the protocol
* library, if fewer bytes than were requested to receive are returned.
* <br><br>
* <b>Example code:</b>
* @code{c}
* int32_t myNetworkRecvImplementation( NetworkContext_t * pNetworkContext,
* void * pBuffer,
* size_t bytesToRecv )
* {
* int32_t bytesReceived = 0;
* bool callTlsRecvFunc = true;
*
* // For a single byte read request, check if data is available on the network.
* if( bytesToRecv == 1 )
* {
* // If no data is available on the network, do not call TLSRecv
* // to avoid blocking for socket timeout.
* if( TLSRecvCount( pNetworkContext->tlsContext ) == 0 )
* {
* callTlsRecvFunc = false;
* }
* }
*
* if( callTlsRecvFunc == true )
* {
* bytesReceived = TLSRecv( pNetworkContext->tlsContext,
* pBuffer,
* bytesToRecv,
* MY_SOCKET_TIMEOUT );
* if( bytesReceived < 0 )
* {
* // If the error code represents a timeout, then the return
* // code should be translated to zero so that the caller
* // can retry the read operation.
* if( bytesReceived == MY_SOCKET_ERROR_TIMEOUT )
* {
* bytesReceived = 0;
* }
* }
* // Handle other cases.
* }
* return bytesReceived;
* }
* @endcode
* <br>
* -# Implementing @ref TransportSend_t<br><br>
* @snippet this define_transportsend
* <br>
* This function is expected to send the bytes, in the given buffer over the
* transport, and return the number of bytes sent.
* In the case of TLS over TCP, @ref TransportSend_t is typically implemented by
* calling the TLS layer function to send data. In case of plaintext TCP
* without TLS, it is typically implemented by calling the TCP layer send
* function. @ref TransportSend_t may be invoked multiple times by the protocol
* library, if fewer bytes than were requested to send are returned.
* <br><br>
* <b>Example code:</b>
* @code{c}
* int32_t myNetworkSendImplementation( NetworkContext_t * pNetworkContext,
* const void * pBuffer,
* size_t bytesToSend )
* {
* int32_t bytesSent = 0;
* bytesSent = TLSSend( pNetworkContext->tlsContext,
* pBuffer,
* bytesToSend,
* MY_SOCKET_TIMEOUT );
*
* // If underlying TCP buffer is full, set the return value to zero
* // so that caller can retry the send operation.
* if( bytesSent == MY_SOCKET_ERROR_BUFFER_FULL )
* {
* bytesSent = 0;
* }
* else if( bytesSent < 0 )
* {
* // Handle socket error.
* }
* // Handle other cases.
*
* return bytesSent;
* }
* @endcode
*/
/**
* @transportstruct
* @typedef NetworkContext_t
* @brief The NetworkContext is an incomplete type. An implementation of this
* interface must define struct NetworkContext for the system requirements.
* This context is passed into the network interface functions.
*/
/* @[define_networkcontext] */
struct NetworkContext;
typedef struct NetworkContext NetworkContext_t;
/* @[define_networkcontext] */
/**
* @transportcallback
* @brief Transport interface for receiving data on the network.
*
* @note It is RECOMMENDED that the transport receive implementation
* does NOT block when requested to read a single byte. A single byte
* read request can be made by the caller to check whether there is a
* new frame available on the network for reading.
* However, the receive implementation MAY block for a timeout period when
* it is requested to read more than 1 byte. This is because once the caller
* is aware that a new frame is available to read on the network, then
* the likelihood of reading more than one byte over the network becomes high.
*
* @param[in] pNetworkContext Implementation-defined network context.
* @param[in] pBuffer Buffer to receive the data into.
* @param[in] bytesToRecv Number of bytes requested from the network.
*
* @return The number of bytes received or a negative value to indicate
* error.
*
* @note If no data is available on the network to read and no error
* has occurred, zero MUST be the return value. A zero return value
* SHOULD represent that the read operation can be retried by calling
* the API function. Zero MUST NOT be returned if a network disconnection
* has occurred.
*/
/* @[define_transportrecv] */
typedef int32_t ( * TransportRecv_t )( NetworkContext_t * pNetworkContext,
void * pBuffer,
size_t bytesToRecv );
/* @[define_transportrecv] */
/**
* @transportcallback
* @brief Transport interface for sending data over the network.
*
* @param[in] pNetworkContext Implementation-defined network context.
* @param[in] pBuffer Buffer containing the bytes to send over the network stack.
* @param[in] bytesToSend Number of bytes to send over the network.
*
* @return The number of bytes sent or a negative value to indicate error.
*
* @note If no data is transmitted over the network due to a full TX buffer and
* no network error has occurred, this MUST return zero as the return value.
* A zero return value SHOULD represent that the send operation can be retried
* by calling the API function. Zero MUST NOT be returned if a network disconnection
* has occurred.
*/
/* @[define_transportsend] */
typedef int32_t ( * TransportSend_t )( NetworkContext_t * pNetworkContext,
const void * pBuffer,
size_t bytesToSend );
/* @[define_transportsend] */
/**
* @transportstruct
* @brief The transport layer interface.
*/
/* @[define_transportinterface] */
typedef struct TransportInterface
{
TransportRecv_t recv; /**< Transport receive interface. */
TransportSend_t send; /**< Transport send interface. */
NetworkContext_t * pNetworkContext; /**< Implementation-defined network context. */
http_parser httpParser; /**< Third-party http-parser context. */
} TransportInterface_t;
/* @[define_transportinterface] */
/* *INDENT-OFF* */
#ifdef __cplusplus
}
#endif
/* *INDENT-ON* */
#endif /* ifndef TRANSPORT_INTERFACE_H_ */

107
arcs-sdk/modules/coreHTTP/test/CMakeLists.txt Normal file
View File

@@ -0,0 +1,107 @@
# Project information.
cmake_minimum_required ( VERSION 3.13.0 )
project ( "coreHTTP unit test"
VERSION 1.0.0
LANGUAGES C )
# Allow the project to be organized into folders.
set_property( GLOBAL PROPERTY USE_FOLDERS ON )
# Use C90.
set( CMAKE_C_STANDARD 90 )
set( CMAKE_C_STANDARD_REQUIRED ON )
# Do not allow in-source build.
if( ${PROJECT_SOURCE_DIR} STREQUAL ${PROJECT_BINARY_DIR} )
message( FATAL_ERROR "In-source build is not allowed. Please build in a separate directory, such as ${PROJECT_SOURCE_DIR}/build." )
endif()
# Set global path variables.
get_filename_component(__MODULE_ROOT_DIR "${CMAKE_CURRENT_LIST_DIR}/.." ABSOLUTE)
set(MODULE_ROOT_DIR ${__MODULE_ROOT_DIR} CACHE INTERNAL "coreHTTP repository root.")
# Configure options to always show in CMake GUI.
option( BUILD_CLONE_SUBMODULES
"Set this to ON to automatically clone any required Git submodules. When OFF, submodules must be manually cloned."
ON )
# Set output directories.
set( CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin )
set( CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib )
set( CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib )
# ====================== Coverity Analysis Configuration =======================
# Include filepaths for source and include.
include( ${MODULE_ROOT_DIR}/httpFilePaths.cmake )
# Target for Coverity analysis that builds the library.
add_library( coverity_analysis
${HTTP_SOURCES} )
# Build HTTP library target without custom config dependency.
target_compile_definitions( coverity_analysis PUBLIC HTTP_DO_NOT_USE_CUSTOM_CONFIG=1 )
# HTTP public include path.
target_include_directories( coverity_analysis PUBLIC ${HTTP_INCLUDE_PUBLIC_DIRS} )
# ===================== Clone needed third-party libraries ======================
# Define an http-paser resource path.
set( HTTP_PARSER_DIR ${MODULE_ROOT_DIR}/source/dependency/3rdparty/http_parser CACHE INTERNAL "http-parser library source directory." )
include( http_parser_build.cmake )
# Check if the http_parser source directory exists.
if( NOT EXISTS ${HTTP_PARSER_DIR}/http_parser.c )
# Attempt to clone http_parser.
if( ${BUILD_CLONE_SUBMODULES} )
clone_http_parser()
else()
message( FATAL_ERROR "The required submodule http_parser does not exist. Either clone it manually, or set BUILD_CLONE_SUBMODULES to 1 to automatically clone it during build." )
endif()
endif()
# ============================ Test configuration ==============================
# Define a CMock resource path.
set( CMOCK_DIR ${MODULE_ROOT_DIR}/test/unit-test/CMock CACHE INTERNAL "CMock library source directory." )
# Include CMock build configuration.
include( unit-test/cmock_build.cmake )
# Check if the CMock source directory exists, and if not present, clone the submodule
# if BUILD_CLONE_SUBMODULES configuration is enabled.
if( NOT EXISTS ${CMOCK_DIR}/src )
# Attempt to clone CMock.
if( ${BUILD_CLONE_SUBMODULES} )
clone_cmock()
else()
message( FATAL_ERROR "The required submodule CMock does not exist. Either clone it manually, or set BUILD_CLONE_SUBMODULES to 1 to automatically clone it during build." )
endif()
endif()
# Add unit test and coverage configuration.
# Use CTest utility for managing test runs. This has to be added BEFORE
# defining test targets with add_test()
enable_testing()
# Add build targets for CMock and Unit, required for unit testing.
add_cmock_targets()
# Add function to enable CMock based tests and coverage.
include( ${MODULE_ROOT_DIR}/tools/cmock/create_test.cmake )
# Include build configuration for unit tests.
add_subdirectory( unit-test )
# ==================== Coverage Analysis configuration ========================
# Add a target for running coverage on tests.
add_custom_target( coverage
COMMAND ${CMAKE_COMMAND} -DCMOCK_DIR=${CMOCK_DIR}
-P ${MODULE_ROOT_DIR}/tools/cmock/coverage.cmake
DEPENDS cmock unity core_http_utest core_http_send_utest
WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
)

4
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/CODE_OF_CONDUCT.md Normal file
View File

@@ -0,0 +1,4 @@
## Code of Conduct
This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).
For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact
opensource-codeofconduct@amazon.com with any additional questions or comments.

61
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,61 @@
# Contributing Guidelines
Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional
documentation, we greatly value feedback and contributions from our community.
Please read through this document before submitting any issues or pull requests to ensure we have all the necessary
information to effectively respond to your bug report or contribution.
## Reporting Bugs/Feature Requests
We welcome you to use the GitHub issue tracker to report bugs or suggest features.
When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already
reported the issue. Please try to include as much information as you can. Details like these are incredibly useful:
* A reproducible test case or series of steps
* The version of our code being used
* Any modifications you've made relevant to the bug
* Anything unusual about your environment or deployment
## Contributing via Pull Requests
Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that:
1. You are working against the latest source on the *master* branch.
2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already.
3. You open an issue to discuss any significant work - we would hate for your time to be wasted.
To send us a pull request, please:
1. Fork the repository.
2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change.
3. Ensure local tests pass.
4. Commit to your fork using clear commit messages.
5. Send us a pull request, answering any default questions in the pull request interface.
6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation.
GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and
[creating a pull request](https://help.github.com/articles/creating-a-pull-request/).
## Finding contributions to work on
Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start.
## Code of Conduct
This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).
For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact
opensource-codeofconduct@amazon.com with any additional questions or comments.
## Security issue notifications
If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.
## Licensing
See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes.

14
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/LICENSE Normal file
View File

@@ -0,0 +1,14 @@
Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
Permission is hereby granted, free of charge, to any person obtaining a copy of this
software and associated documentation files (the "Software"), to deal in the Software
without restriction, including without limitation the rights to use, copy, modify,
merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

1
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/NOTICE Normal file
View File

@@ -0,0 +1 @@
Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.

102
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/README.md Normal file
View File

@@ -0,0 +1,102 @@
# AWS Templates for CBMC Proofs
This repository is a "starter kit" for writing CBMC proofs.
[CBMC](https://www.cprover.org/cbmc/)
is a model checker for C code that can prove that the assertions in your code
are never violated and that your code is free
of security vulnerabilites like buffer overflow. In this starter kit,
* one script ([setup.py](https://github.com/awslabs/aws-templates-for-cbmc-proofs/blob/master/scripts/setup.py))
installs into your repository a few directories containing code, templates, and Makefiles that will be
useful for every proof you write, and
* one script ([setup-proof.py](https://github.com/awslabs/aws-templates-for-cbmc-proofs/blob/master/scripts/setup-proof.py))
installs, for a particular proof, a single directory containing skeletons of all the files you will need
to write that proof.
The [starter kit wiki](https://github.com/awslabs/aws-templates-for-cbmc-proofs/wiki) is the
primary documentation for the starter kit. It includes tips on how to plan your proof,
how to write a good proof, and how to debug a failed proof.
It also includes
[installation instructions](https://github.com/awslabs/aws-templates-for-cbmc-proofs/wiki/Installation)
for installing the tools [CBMC](https://github.com/diffblue/cbmc) and
[CBMC viewer](https://github.com/awslabs/aws-viewer-for-cbmc) that you will need to use the starter kit.
You should install these tools now.
What follows is quick start guide to using the starter kit.
It sketchs how to install the starter kit and how to start a new proof.
## Installing the starter kit
If you are working on a new project that does not have the starter kit installed,
you will need to install it from scratch.
If you are working on an existing project that already has the starter kit installed,
you will need to include the starter kit when you clone your project for the first time.
### Using the starter kit on a new project
If the starter kit is not already installed in your project, you must
submodule the starter kit into your project and install it.
To do this, clone your repository as usual, change directory into your
repository, and perform the following steps:
* Choose the path to the source root (eg, /usr/project)
* Choose the path to a directory under the source root that should hold
the infrastructure (eg, /usr/project/cbmc)
* Submodule the AWS-templates-for-CBMC repository into this directory (eg,
/usr/project/cbmc/aws-templates-for-cbmc)
```
cd /usr/project/cbmc
git submodule add https://github.com/awslabs/aws-templates-for-cbmc-proofs.git aws-templates-for-cbmc-proofs
```
* Use the script `aws-templates-for-cbmc-proofs/scripts/setup.py` to
setup the standard directory structure for CBMC proof.
```
cd /usr/project/cbmc
python3 aws-templates-for-cbmc-proofs/scripts/setup.py
```
The script will ask for the path to the source root `/usr/project`.
The script will install four directories into `/usr/project/cbmc`:
* include: contains include files for the proofs
* sources: contains source code for the proofs
* stubs: contains stubs for the proofs
* proofs: contains the proofs themselves (the proof root).
See the [setup wiki page](https://github.com/awslabs/aws-templates-for-cbmc-proofs/wiki/CBMC-starter-kit-setup-script) for more details on how to use the script.
Commit these changes, and you are ready to go.
### Using the starter kit on an existing project
If the starter kit is already installed in your project,
you just have to remember to include the stater kit whenever you clone your repository.
To do this, clone your repository as usual, change directory into your
repository, and run the command:
````
git submodule update --init --checkout --recursive
````
## Starting a new proof
Once the starter kit is installed, you start a new proof by running a
proof setup script.
* Change to a directory under the proof root (/usr/project/cbmc/proofs)
* Run the script `../aws-templates-for-cbmc/scripts/setup-proof.py` and give
* The name of the function under test.
* The path to the source file defining the function under test.
* The path to the source root (eg, /usr/project).
The script will create a directory named for the function, and will
install files needed to run the proof. Now you can cut and paste into
the files, and type `make` to run and debug the proof.
See the
[proof setup wiki page](https://github.com/awslabs/aws-templates-for-cbmc-proofs/wiki/CBMC-starter-kit-setup-proof-script)
for more details on how to use the script and the files it installs.
## Security
See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.
## License
This project is licensed under the Apache-2.0 License.

184
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/remove-apache-license-identifier.py Executable file
View File

@@ -0,0 +1,184 @@
#!/usr/bin/env python3
import argparse
import json
import logging
import os
import re
import shutil
import subprocess
import sys
################################################################
# Command line arguments
def create_parser(desc, args, epilog=None):
default_args = [
{
"flag": "--verbose",
"action": "store_true",
"help": "Verbose output"
},
{
"flag": "--debug",
"action": "store_true",
"help": "Debug output"
}
]
args.extend(default_args)
parser = argparse.ArgumentParser(description=desc, epilog=epilog)
for arg in args:
flag = arg.pop('flag')
parser.add_argument(flag, **arg)
return parser
def parser():
desc = "Remove Apache references from files copied from the CBMC starter kit"
args = [
{
"flag": "--proofdir",
"help": "Root of the proof subtree (default: %(default)s)",
"default": ".",
},
{
"flag": "--remove",
"action": "store_true",
"help": "Remove Apache references from files under PROOFDIR (otherwise just list them)"
},
]
epilog = """
The CBMC starter kit was originally released under the Apache
license. All files in the starter kit contained references to the
Apache license. The starter kit installation scripts copied files
from the stater kit into the project repository. This became an
issue when the project repository was released under a different
license. This script removes all references to the Apache license
from the files copied into the project repository from the starter
kit.
"""
return create_parser(desc, args, epilog)
def configure_logging(args):
# Logging is configured by the first invocation of logging.basicConfig
fmt = '%(levelname)s: %(message)s'
if args.debug:
logging.basicConfig(level=logging.DEBUG, format=fmt)
if args.verbose:
logging.basicConfig(level=logging.INFO, format=fmt)
logging.basicConfig(format=fmt)
################################################################
# Shell out commands
def run(cmd, cwd=None, encoding=None):
"""Run a command in a subshell and return the standard output.
Run the command cmd in the directory cwd and use encoding to
decode the standard output.
"""
kwds = {
'cwd': cwd,
'stdout': subprocess.PIPE,
'stderr': subprocess.PIPE,
'text': True,
}
if sys.version_info >= (3, 6): # encoding introduced in Python 3.6
kwds['encoding'] = encoding
logging.debug('Running "%s" in %s', ' '.join(cmd), cwd)
result = subprocess.run(cmd, **kwds, check=False)
if result.returncode:
logging.debug('Failed command: %s', ' '.join(cmd))
logging.debug('Failed return code: %s', result.returncode)
logging.debug('Failed stdout: %s', result.stdout.strip())
logging.debug('Failed stderr: %s', result.stderr.strip())
return []
# Remove line continuations before splitting stdout into lines
# Running command with text=True converts line endings to \n in stdout
lines = result.stdout.replace('\\\n', ' ').splitlines()
return [strip_whitespace(line) for line in lines]
def strip_whitespace(string):
return re.sub(r'\s+', ' ', string).strip()
################################################################
def maybe_a_copied_file(path):
copied_files = [
'Makefile',
'Makefile-project-defines',
'Makefile-project-targets',
'Makefile-project-testing',
]
return os.path.basename(path) in copied_files
def find_apache_references(proofdir=None):
paths = run(['git', 'grep', '-l', 'Apache', '.'], cwd=proofdir)
paths = [os.path.normpath(os.path.join(proofdir, path)) for path in paths]
paths = [path for path in paths if not os.path.islink(path)]
return sorted(paths)
def remove_apache_reference(path, extension='backup'):
apache_references = [
'# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.',
'# SPDX-License-Identifier: Apache-2.0'
]
backup = path + '.' + extension
shutil.move(path, backup)
with open(backup) as infile, open(path, "w") as outfile:
removed = False
for line in infile:
if strip_whitespace(line) in apache_references:
removed = True
logging.debug('Deleted Apache reference in %s: %s',
path, strip_whitespace(line))
continue
outfile.write(line)
if removed:
logging.info('Deleted Apache reference in %s', path)
return True
return False
def remove_apache_references(paths):
removed = False
for path in paths:
if not maybe_a_copied_file(path):
logging.debug('Skipping %s', path)
continue
logging.debug('Updating %s', path)
removed = remove_apache_reference(path) or removed
return removed
################################################################
def main():
args = parser().parse_args()
configure_logging(args)
paths = find_apache_references(args.proofdir)
if not args.remove:
if paths:
print("The following files contain references to the Apache license:")
for path in paths:
print(f" {path}")
script = os.path.basename(sys.argv[0])
print(f"Remove Apache references from these files with '{script} --remove'")
exit(0)
remove_apache_references(paths)
paths = find_apache_references(args.proofdir)
if paths:
logging.warning("Files left unchanged contain Apache references: %s", ', '.join(paths))
exit(1)
exit(0)
if __name__ == "__main__":
main()

70
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/scripts/setup-proof.py Executable file
View File

@@ -0,0 +1,70 @@
#!/usr/bin/env python3
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0
"""Set up a CBMC proof."""
import logging
import os
import shutil
import util
def proof_template_filenames():
directory = os.path.join(util.templates_root(), util.PROOF_TEMPLATES)
return os.listdir(directory)
def read_proof_template(filename):
directory = os.path.join(util.templates_root(), util.PROOF_TEMPLATES)
with open(os.path.join(directory, filename)) as data:
return data.read().splitlines()
def write_proof_template(lines, filename, directory):
with open(os.path.join(directory, filename), "w") as data:
data.writelines(line + '\n' for line in lines)
def rename_proof_harness(function, directory):
shutil.move(os.path.join(directory, "FUNCTION_harness.c"),
os.path.join(directory, "{}_harness.c".format(function)))
def patch_function_name(lines, function):
return [line.replace("<__FUNCTION_NAME__>", function) for line in lines]
def patch_path_to_makefile(lines, proof_root, proof_dir):
path = os.path.relpath(proof_root, proof_dir)
return [line.replace("<__PATH_TO_MAKEFILE__>", path) for line in lines]
def patch_path_to_proof_root(lines, proof_root, source_root):
path = os.path.relpath(proof_root, source_root)
return [line.replace("<__PATH_TO_PROOF_ROOT__>", path) for line in lines]
def patch_path_to_source_file(lines, source_file, source_root):
path = os.path.relpath(source_file, source_root)
return [line.replace("<__PATH_TO_SOURCE_FILE__>", path) for line in lines]
def main():
"""Set up CBMC proof."""
logging.basicConfig(format='%(levelname)s: %(message)s')
function = util.read_function_name()
source_file = util.read_source_path()
source_root = util.read_source_root_path()
proof_root = util.read_proof_root_path()
proof_dir = os.path.abspath(function)
os.mkdir(proof_dir)
for filename in proof_template_filenames():
lines = read_proof_template(filename)
lines = patch_function_name(lines, function)
lines = patch_path_to_makefile(lines, proof_root, proof_dir)
lines = patch_path_to_proof_root(lines, proof_root, source_root)
lines = patch_path_to_source_file(lines, source_file, source_root)
write_proof_template(lines, filename, proof_dir)
rename_proof_harness(function, proof_dir)
if __name__ == "__main__":
main()

74
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/scripts/setup.py Executable file
View File

@@ -0,0 +1,74 @@
#!/usr/bin/env python3
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0
"""Set up the CBMC proof instrastructure."""
import logging
import os
import util
SRCDIR_TEXT = """
# Absolute path to the root of the source tree.
#
SRCDIR ?= $(abspath $(PROOF_ROOT)/{})
"""
LITANI_TEXT = """
# Absolute path to the litani script.
#
LITANI ?= $(abspath $(PROOF_ROOT)/{})
"""
PROJECT_TEXT = """
# Name of this proof project, displayed in proof reports. For example,
# "s2n" or "Amazon FreeRTOS". For projects with multiple proof roots,
# this may be overridden on the command-line to Make, for example
#
# make PROJECT_NAME="FreeRTOS MQTT" report
#
PROJECT_NAME = "{}"
"""
def create_makefile_template_defines(
proof_root, source_root, litani, project_name):
"""Create Makefile-template-defines in the proof root."""
makefile = os.path.join(proof_root, "Makefile-template-defines")
if os.path.exists(makefile):
logging.warning("Overwriting %s", makefile)
with open(makefile, "w") as fileobj:
print(SRCDIR_TEXT.format(os.path.relpath(source_root, proof_root)),
file=fileobj)
print(LITANI_TEXT.format(os.path.relpath(litani, proof_root)),
file=fileobj)
print(PROJECT_TEXT.format(project_name), file=fileobj)
def main():
"""Set up the CBMC proof infrastructure."""
logging.basicConfig(format='%(levelname)s: %(message)s')
source_root = util.read_source_root_path()
# the script is being run in the cbmc root
cbmc_root = os.path.abspath('.')
# the script is creating the proof root
proof_root = os.path.abspath('proofs')
# the script is linking to the litani script within the litani submodule
litani = util.read_litani_path()
# the name of the project used in project verification reports
project_name = util.read_project_name()
util.copy_repository_templates(cbmc_root)
create_makefile_template_defines(
proof_root, source_root, litani, project_name)
if __name__ == "__main__":
main()

130
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/scripts/util.py Normal file
View File

@@ -0,0 +1,130 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0
"""Methods of manipulating the templates repository."""
import logging
import os
import shutil
REPOSITORY_TEMPLATES = "template-for-repository"
PROOF_TEMPLATES = "template-for-proof"
PROOF_DIR = "proofs"
# There are some files that we should copy to the project repository rather than
# symlinking. This is because users are expected to modify these files. If the
# files were symlinks, then modifying them would dirty up this submodule, which
# would prevent project owners from cleanly updating it.
COPY_INSTEAD = [
"Makefile-project-defines",
"Makefile-project-targets",
"Makefile-project-testing",
".gitignore",
]
################################################################
def script_dir():
"""Directory containing setup scripts."""
return os.path.dirname(os.path.abspath(__file__))
def templates_root():
"""Directory containing the AWS-templates-for-CBMC repository."""
return os.path.dirname(script_dir())
################################################################
# Read configuration information from the standard input
def read_from_stdin():
return input().strip()
def read_path_from_stdin(description):
print("What is the path to {}? ".format(description), end="")
return os.path.abspath(os.path.expanduser(read_from_stdin()))
def read_source_root_path():
return read_path_from_stdin("the source root")
def read_proof_root_path():
return read_path_from_stdin("the 'proofs' directory (usually '.')")
def read_litani_path():
return read_path_from_stdin("the litani script")
def read_source_path():
return read_path_from_stdin("the source file defining the function")
def read_function_name():
print("What is the function name? ", end="")
return read_from_stdin()
def read_project_name():
print("What is the project name? ", end="")
return read_from_stdin()
################################################################
def files_under_root(root):
"""The list of files in the filesystem under root."""
cwd = os.getcwd()
try:
os.chdir(root)
return [os.path.join(path, name)
for path, _, files in os.walk('.') for name in files]
finally:
os.chdir(cwd)
def link_files(name, src, dst):
"""Link file dst/name to file src/name, return number skipped"""
src_name = os.path.normpath(os.path.join(src, name))
dst_name = os.path.normpath(os.path.join(dst, name))
os.makedirs(os.path.dirname(dst_name), exist_ok=True)
src_link = os.path.relpath(src_name, os.path.dirname(dst_name))
if os.path.basename(name) in COPY_INSTEAD:
install_method = ("copy", shutil.copyfile)
src_link = src_name
else:
install_method = ("symlink", os.symlink)
if os.path.exists(dst_name):
logging.warning("Skipping %s %s -> %s: file exists",
install_method[0], name, src_link)
return 1
logging.warning(
"Creating %s %s -> %s", install_method[0], name, src_link)
install_method[1](src_link, dst_name)
return 0
def copy_directory_contents(src, dst):
"""Link the contents of one directory into another."""
src = os.path.normpath(src)
dst = os.path.normpath(dst)
assert os.path.isdir(src)
assert os.path.isdir(dst)
skipped = 0
for name in files_under_root(src):
name = os.path.normpath(name)
skipped += link_files(name, src, dst)
if skipped:
logging.warning("To overwrite a skipped file, "
"delete the file and rerun the script.")
def copy_repository_templates(cbmc_root):
"""Copy the files in the repository template into the CBMC root."""
copy_directory_contents(os.path.join(templates_root(),
REPOSITORY_TEMPLATES),
cbmc_root)

25
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-proof/FUNCTION_harness.c Normal file
View File

@@ -0,0 +1,25 @@
// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
// SPDX-License-Identifier: MIT-0
/*
* Insert copyright notice
*/
/**
* @file <__FUNCTION_NAME__>_harness.c
* @brief Implements the proof harness for <__FUNCTION_NAME__> function.
*/
/*
* Insert project header files that
* - include the declaration of the function
* - include the types needed to declare function arguments
*/
void harness()
{
/* Insert argument declarations */
<__FUNCTION_NAME__>( /* Insert arguments */ );
}

23
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-proof/Makefile Normal file
View File

@@ -0,0 +1,23 @@
HARNESS_ENTRY = harness
HARNESS_FILE = <__FUNCTION_NAME__>_harness
# This should be a unique identifier for this proof, and will appear on the
# Litani dashboard. It can be human-readable and contain spaces if you wish.
PROOF_UID = <__FUNCTION_NAME__>
DEFINES +=
INCLUDES +=
REMOVE_FUNCTION_BODY +=
UNWINDSET +=
PROOF_SOURCES += $(PROOFDIR)/$(HARNESS_FILE).c
PROJECT_SOURCES += $(SRCDIR)/<__PATH_TO_SOURCE_FILE__>
# If this proof is found to consume huge amounts of RAM, you can set the
# EXPENSIVE variable. With new enough versions of the proof tools, this will
# restrict the number of EXPENSIVE CBMC jobs running at once. See the
# documentation in Makefile.common under the "Job Pools" heading for details.
# EXPENSIVE = true
include <__PATH_TO_MAKEFILE__>/Makefile.common

20
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-proof/README.md Normal file
View File

@@ -0,0 +1,20 @@
<__FUNCTION_NAME__> proof
==============
This directory contains a memory safety proof for <__FUNCTION_NAME__>.
To run the proof.
-------------
* Add `cbmc`, `goto-cc`, `goto-instrument`, `goto-analyzer`, and `cbmc-viewer`
to your path.
* Run `make`.
* Open html/index.html in a web browser.
To use [`arpa`](https://github.com/awslabs/aws-proof-build-assistant) to simplify writing Makefiles.
-------------
* Run `make arpa` to generate a Makefile.arpa that contains relevant build information for the proof.
* Use Makefile.arpa as the starting point for your proof Makefile by:
1. Modifying Makefile.arpa (if required).
2. Including Makefile.arpa into the existing proof Makefile (add `sinclude Makefile.arpa` at the bottom of the Makefile, right before `include ../Makefile.common`).

1
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-proof/cbmc-proof.txt Normal file
View File

@@ -0,0 +1 @@
# This file marks this directory as containing a CBMC proof.

7
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-proof/cbmc-viewer.json Normal file
View File

@@ -0,0 +1,7 @@
{ "expected-missing-functions":
[
],
"proof-name": "<__FUNCTION_NAME__>",
"proof-root": "<__PATH_TO_PROOF_ROOT__>"
}

6
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/include/README.md Normal file
View File

@@ -0,0 +1,6 @@
CBMC proof include files
========================
This directory contains include files written for CBMC proof. It is
common to write some code to model aspects of the system under test,
and the header files for this code go here.

13
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/README.md Normal file
View File

@@ -0,0 +1,13 @@
## Negative Tests
This directory contains negative checks to ensure that CBMC CI jobs are run
with the complete set of property-checking flags
(see `CHECKFLAGS` in [Makefile.common](../proofs/Makefile.common))
which we consider to be part of the best practice.
To enable these tests in CI jobs,
copy this (`negative_tests`) directory into `../proofs`.
If a property-checking flag is not used used by your project,
you might want to disable the corresponding negative test.
To do so, simply delete the particular test directory from `../proofs/negative_tests`.

13
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/assert/Makefile Normal file
View File

@@ -0,0 +1,13 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0.
EXPECTED = FAILED
HARNESS_ENTRY = assert_harness
HARNESS_FILE = $(HARNESS_ENTRY).c
PROOF_SOURCES += $(HARNESS_FILE)
PROJECT_SOURCES += $(HARNESS_FILE)
include ../Makefile.common

13
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/assert/assert_harness.c Normal file
View File

@@ -0,0 +1,13 @@
/**
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: MIT-0.
*/
/**
* A basic negative assertion should fail
* if CBMC was run at all.
*/
void assert_harness() {
int lhs, rhs;
assert(lhs == rhs);
}

13
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/bounds_check/Makefile Normal file
View File

@@ -0,0 +1,13 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0.
EXPECTED = FAILED
HARNESS_ENTRY = bounds_check_harness
HARNESS_FILE = $(HARNESS_ENTRY).c
PROOF_SOURCES += $(HARNESS_FILE)
PROJECT_SOURCES += $(HARNESS_FILE)
include ../Makefile.common

16
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/bounds_check/bounds_check_harness.c Normal file
View File

@@ -0,0 +1,16 @@
/**
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: MIT-0.
*/
#include <stddef.h>
/**
* A negative test for --bounds-check flag
*/
void bounds_check_harness() {
char test[10];
size_t index;
char ch;
test[index] = ch;
}

13
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/conversion_check/Makefile Normal file
View File

@@ -0,0 +1,13 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0.
EXPECTED = FAILED
HARNESS_ENTRY = conversion_check_harness
HARNESS_FILE = $(HARNESS_ENTRY).c
PROOF_SOURCES += $(HARNESS_FILE)
PROJECT_SOURCES += $(HARNESS_FILE)
include ../Makefile.common

14
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/conversion_check/conversion_check_harness.c Normal file
View File

@@ -0,0 +1,14 @@
/**
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: MIT-0.
*/
#include <stdint.h>
/**
* A negative test for --conversion-check flag
*/
void conversion_check_harness() {
uint64_t src;
uint32_t dst = src;
}

13
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/div_by_zero_check/Makefile Normal file
View File

@@ -0,0 +1,13 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0.
EXPECTED = FAILED
HARNESS_ENTRY = div_by_zero_check_harness
HARNESS_FILE = $(HARNESS_ENTRY).c
PROOF_SOURCES += $(HARNESS_FILE)
PROJECT_SOURCES += $(HARNESS_FILE)
include ../Makefile.common

12
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/div_by_zero_check/div_by_zero_check_harness.c Normal file
View File

@@ -0,0 +1,12 @@
/**
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: MIT-0.
*/
/**
* A negative test for --div-by-zero-check flag
*/
void div_by_zero_check_harness() {
int num, den;
int div = num / den;
}

13
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/float_overflow_check/Makefile Normal file
View File

@@ -0,0 +1,13 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0.
EXPECTED = FAILED
HARNESS_ENTRY = float_overflow_check_harness
HARNESS_FILE = $(HARNESS_ENTRY).c
PROOF_SOURCES += $(HARNESS_FILE)
PROJECT_SOURCES += $(HARNESS_FILE)
include ../Makefile.common

12
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/float_overflow_check/float_overflow_check_harness.c Normal file
View File

@@ -0,0 +1,12 @@
/**
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: MIT-0.
*/
/**
* A negative test for --float-overflow-check flag
*/
void float_overflow_check_harness() {
float overflow, increment;
overflow += increment;
}

13
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/float_underflow_check/Makefile Normal file
View File

@@ -0,0 +1,13 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0.
EXPECTED = FAILED
HARNESS_ENTRY = float_underflow_check_harness
HARNESS_FILE = $(HARNESS_ENTRY).c
PROOF_SOURCES += $(HARNESS_FILE)
PROJECT_SOURCES += $(HARNESS_FILE)
include ../Makefile.common

12
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/float_underflow_check/float_underflow_check_harness.c Normal file
View File

@@ -0,0 +1,12 @@
/**
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: MIT-0.
*/
/**
* A negative test for --float-overflow-check flag
*/
void float_underflow_check_harness() {
float underflow, increment;
underflow -= increment;
}

13
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/nan_check/Makefile Normal file
View File

@@ -0,0 +1,13 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0.
EXPECTED = FAILED
HARNESS_ENTRY = nan_check_harness
HARNESS_FILE = $(HARNESS_ENTRY).c
PROOF_SOURCES += $(HARNESS_FILE)
PROJECT_SOURCES += $(HARNESS_FILE)
include ../Makefile.common

12
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/nan_check/nan_check_harness.c Normal file
View File

@@ -0,0 +1,12 @@
/**
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: MIT-0.
*/
/**
* A negative test for --nan-check flag
*/
void nan_check_harness() {
float nan;
nan = nan / nan;
}

13
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/pointer_check/Makefile Normal file
View File

@@ -0,0 +1,13 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0.
EXPECTED = FAILED
HARNESS_ENTRY = pointer_check_harness
HARNESS_FILE = $(HARNESS_ENTRY).c
PROOF_SOURCES += $(HARNESS_FILE)
PROJECT_SOURCES += $(HARNESS_FILE)
include ../Makefile.common

12
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/pointer_check/pointer_check_harness.c Normal file
View File

@@ -0,0 +1,12 @@
/**
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: MIT-0.
*/
/**
* A negative test for --pointer-check flag
*/
void pointer_check_harness() {
int *src;
int test = *src;
}

13
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/pointer_overflow_check/Makefile Normal file
View File

@@ -0,0 +1,13 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0.
EXPECTED = FAILED
HARNESS_ENTRY = pointer_overflow_check_harness
HARNESS_FILE = $(HARNESS_ENTRY).c
PROOF_SOURCES += $(HARNESS_FILE)
PROJECT_SOURCES += $(HARNESS_FILE)
include ../Makefile.common

15
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/pointer_overflow_check/pointer_overflow_check_harness.c Normal file
View File

@@ -0,0 +1,15 @@
/**
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: MIT-0.
*/
#include <stddef.h>
/**
* A negative test for --pointer-overflow-check flag
*/
void pointer_overflow_check_harness() {
size_t offset;
char *pointer;
pointer += offset;
}

15
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/pointer_primitive_check/Makefile Normal file
View File

@@ -0,0 +1,15 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0.
CHECKFLAGS += --pointer-primitive-check
EXPECTED = FAILED
HARNESS_ENTRY = pointer_primitive_check_harness
HARNESS_FILE = $(HARNESS_ENTRY).c
PROOF_SOURCES += $(HARNESS_FILE)
PROJECT_SOURCES += $(HARNESS_FILE)
include ../Makefile.common

12
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/pointer_primitive_check/pointer_primitive_check_harness.c Normal file
View File

@@ -0,0 +1,12 @@
/**
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: MIT-0.
*/
/**
* A negative test for --pointer-primitive-check flag
*/
void pointer_primitive_check_harness() {
char *pointer;
assert(__CPROVER_r_ok(pointer, 10));
}

13
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/pointer_underflow_check/Makefile Normal file
View File

@@ -0,0 +1,13 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0.
EXPECTED = FAILED
HARNESS_ENTRY = pointer_underflow_check_harness
HARNESS_FILE = $(HARNESS_ENTRY).c
PROOF_SOURCES += $(HARNESS_FILE)
PROJECT_SOURCES += $(HARNESS_FILE)
include ../Makefile.common

15
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/pointer_underflow_check/pointer_underflow_check_harness.c Normal file
View File

@@ -0,0 +1,15 @@
/**
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: MIT-0.
*/
#include <stddef.h>
/**
* A negative test for --pointer-overflow-check flag
*/
void pointer_underflow_check_harness() {
size_t offset;
char *pointer;
pointer -= offset;
}

13
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/signed_overflow_check/Makefile Normal file
View File

@@ -0,0 +1,13 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0.
EXPECTED = FAILED
HARNESS_ENTRY = signed_overflow_check_harness
HARNESS_FILE = $(HARNESS_ENTRY).c
PROOF_SOURCES += $(HARNESS_FILE)
PROJECT_SOURCES += $(HARNESS_FILE)
include ../Makefile.common

12
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/signed_overflow_check/signed_overflow_check_harness.c Normal file
View File

@@ -0,0 +1,12 @@
/**
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: MIT-0.
*/
/**
* A negative test for --signed-overflow-check flag
*/
void signed_overflow_check_harness() {
int overflow, offset;
overflow += offset;
}

13
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/signed_underflow_check/Makefile Normal file
View File

@@ -0,0 +1,13 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0.
EXPECTED = FAILED
HARNESS_ENTRY = signed_underflow_check_harness
HARNESS_FILE = $(HARNESS_ENTRY).c
PROOF_SOURCES += $(HARNESS_FILE)
PROJECT_SOURCES += $(HARNESS_FILE)
include ../Makefile.common

12
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/signed_underflow_check/signed_underflow_check_harness.c Normal file
View File

@@ -0,0 +1,12 @@
/**
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: MIT-0.
*/
/**
* A negative test for --signed-overflow-check flag
*/
void signed_underflow_check_harness() {
int underflow, offset;
underflow -= offset;
}

13
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/undefined_shift_check/Makefile Normal file
View File

@@ -0,0 +1,13 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0.
EXPECTED = FAILED
HARNESS_ENTRY = undefined_shift_check_harness
HARNESS_FILE = $(HARNESS_ENTRY).c
PROOF_SOURCES += $(HARNESS_FILE)
PROJECT_SOURCES += $(HARNESS_FILE)
include ../Makefile.common

13
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/undefined_shift_check/undefined_shift_check_harness.c Normal file
View File

@@ -0,0 +1,13 @@
/**
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: MIT-0.
*/
/**
* A negative test for --undefined-shift-check flag
*/
void cbmc_ensure__undefined_shift_check_harness() {
int base, shift;
base <<= shift;
base >>= shift;
}

13
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/unsigned_overflow_check/Makefile Normal file
View File

@@ -0,0 +1,13 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0.
EXPECTED = FAILED
HARNESS_ENTRY = unsigned_overflow_check_harness
HARNESS_FILE = $(HARNESS_ENTRY).c
PROOF_SOURCES += $(HARNESS_FILE)
PROJECT_SOURCES += $(HARNESS_FILE)
include ../Makefile.common

12
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/unsigned_overflow_check/unsigned_overflow_check_harness.c Normal file
View File

@@ -0,0 +1,12 @@
/**
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: MIT-0.
*/
/**
* A negative test for --unsigned-overflow-check flag
*/
void unsigned_overflow_check_harness() {
unsigned overflow, offset;
overflow += offset;
}

13
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/unsigned_underflow_check/Makefile Normal file
View File

@@ -0,0 +1,13 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0.
EXPECTED = FAILED
HARNESS_ENTRY = unsigned_underflow_check_harness
HARNESS_FILE = $(HARNESS_ENTRY).c
PROOF_SOURCES += $(HARNESS_FILE)
PROJECT_SOURCES += $(HARNESS_FILE)
include ../Makefile.common

12
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/negative_tests/unsigned_underflow_check/unsigned_underflow_check_harness.c Normal file
View File

@@ -0,0 +1,12 @@
/**
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: MIT-0.
*/
/**
* A negative test for --unsigned-overflow-check flag
*/
void unsigned_underflow_check_harness() {
unsigned underflow, offset;
underflow -= offset;
}

34
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/proofs/Makefile-project-defines Normal file
View File

@@ -0,0 +1,34 @@
# -*- mode: makefile -*-
# The first line sets the emacs major mode to Makefile
################################################################
# Use this file to give project-specific definitions of the command
# line arguments to pass to CBMC tools like goto-cc to build the goto
# binaries and cbmc to do the property and coverage checking.
#
# Use this file to override most default definitions of variables in
# Makefile.common.
################################################################
# Flags to pass to goto-cc for compilation (typically those passed to gcc -c)
# COMPILE_FLAGS =
# Flags to pass to goto-cc for linking (typically those passed to gcc)
# LINK_FLAGS =
# Preprocessor include paths -I...
# Consider adding
# INCLUDES += -I$(CBMC_ROOT)/include
# You will want to decide what order that comes in relative to the other
# include directories in your project.
#
# INCLUDES =
# Preprocessor definitions -D...
# DEFINES =
# Path to arpa executable
# ARPA =
# Flags to pass to cmake for building the project
# ARPA_CMAKE_FLAGS =

7
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/proofs/Makefile-project-targets Normal file
View File

@@ -0,0 +1,7 @@
# -*- mode: makefile -*-
# The first line sets the emacs major mode to Makefile
################################################################
# Use this file to give project-specific targets, including targets
# that may depend on targets defined in Makefile.common.
################################################################

8
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/proofs/Makefile-project-testing Normal file
View File

@@ -0,0 +1,8 @@
# -*- mode: makefile -*-
# The first line sets the emacs major mode to Makefile
################################################################
# Use this file to define project-specific targets and definitions for
# unit testing or continuous integration that may depend on targets
# defined in Makefile.common
################################################################

923
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/proofs/Makefile.common Normal file
View File

@@ -0,0 +1,923 @@
# -*- mode: makefile -*-
# The first line sets the emacs major mode to Makefile
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0
################################################################
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"). You
# may not use this file except in compliance with the License. A copy
# of the License is located at
#
# http://aws.amazon.com/apache2.0/
#
# or in the "license" file accompanying this file. This file is
# distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
# ANY KIND, either express or implied. See the License for the
# specific language governing permissions and limitations under the
# License.
################################################################
# This file Makefile.common defines the basic work flow for cbmc proofs.
#
# The intention is that the goal of your project is to write proofs
# for a collection of functions in a source tree.
#
# To use this file
# 1. Edit the variable definitions in Section I below as appropriate for
# your project, your proofs, and your source tree.
# 2. For each function for which you are writing a proof,
# a. Create a subdirectory <DIR>.
# b. Write a proof harness (a function) with the name <HARNESS_ENTRY>
# in a file with the name <DIR>/<HARNESS_FILE>.c
# c. Write a makefile with the name <DIR>/Makefile that looks
# something like
#
# HARNESS_FILE=<HARNESS_FILE>
# HARNESS_ENTRY=<HARNESS_ENTRY>
# PROOF_UID=<PROOF_UID>
#
# PROJECT_SOURCES += $(SRCDIR)/libraries/api_1.c
# PROJECT_SOURCES += $(SRCDIR)/libraries/api_2.c
#
# PROOF_SOURCES += $(PROOFDIR)/harness.c
# PROOF_SOURCES += $(SRCDIR)/cbmc/proofs/stub_a.c
# PROOF_SOURCES += $(SRCDIR)/cbmc/proofs/stub_b.c
#
# UNWINDSET += foo.0:3
# UNWINDSET += bar.1:6
#
# REMOVE_FUNCTION_BODY += api_stub_a
# REMOVE_FUNCTION_BODY += api_stub_b
#
# DEFINES = -DDEBUG=0
#
# include ../Makefile.common
#
# d. Change directory to <DIR> and run make
#
# Dependency handling in this file may not be perfect. Consider
# running "make clean" or "make veryclean" before "make report" if you
# get results that are hard to explain.
SHELL=/bin/bash
default: report
################################################################
################################################################
## Section I: This section gives common variable definitions.
##
## Feel free to edit these definitions for your project.
##
## Definitions specific to a proof (generally definitions defined
## below with ?= like PROJECT_SOURCES listing the project source files
## required by the proof) should be defined in the proof Makefile.
##
## Remember that this Makefile is intended to be included from the
## Makefile in your proof directory, so all relative pathnames should
## be relative to your proof directory.
##
# Absolute path to the directory containing this Makefile.common
# See https://ftp.gnu.org/old-gnu/Manuals/make-3.80/html_node/make_17.html
#
# Note: We compute the absolute paths to the makefiles in MAKEFILE_LIST
# before we filter the list of makefiles for %/Makefile.common.
# Otherwise an invocation of the form "make -f Makefile.common" will set
# MAKEFILE_LIST to "Makefile.common" which will fail to match the
# pattern %/Makefile.common.
#
MAKEFILE_PATHS = $(foreach makefile,$(MAKEFILE_LIST),$(abspath $(makefile)))
PROOF_ROOT = $(dir $(filter %/Makefile.common,$(MAKEFILE_PATHS)))
CBMC_ROOT = $(shell dirname $(PROOF_ROOT))
PROOF_STUB = $(CBMC_ROOT)/stubs
PROOF_SOURCE = $(CBMC_ROOT)/sources
# Project-specific definitions to override default definitions below
# * Makefile-project-defines will never be overwritten
# * Makefile-template-defines will be overwritten each time the
# proof templates are updated
sinclude $(PROOF_ROOT)/Makefile-project-defines
sinclude $(PROOF_ROOT)/Makefile-template-defines
# SRCDIR is the path to the root of the source tree
SRCDIR ?= $(abspath ../..)
# PROOFDIR is the path to the directory containing the proof harness
PROOFDIR ?= $(abspath .)
# Path to the root of the cbmc project.
#
# Projects generally have a directory $(CBMCDIR) with subdirectories
# $(CBMCDIR)/proofs containing the proofs and maybe $(CBMCDIR)/stubs
# containing the stubs used in the proof. This Makefile is generally
# at $(CBMCDIR)/proofs/Makefile.common.
CBMCDIR ?= $(PROOF_ROOT)/cbmc
# Default CBMC flags used for property checking and coverage checking
CBMCFLAGS += --unwind 1 $(CBMC_UNWINDSET) --flush
# Do property checking with the external SAT solver given by
# EXTERNAL_SAT_SOLVER. Do coverage checking with the default solver,
# since coverage checking requires the use of an incremental solver.
# The EXTERNAL_SAT_SOLVER variable is typically set (if it is at all)
# as an environment variable or as a makefile variable in
# Makefile-project-defines.
#
# For a particular proof, if the default solver is faster, do property
# checking with the default solver by including this definition in the
# proof Makefile:
# USE_EXTERNAL_SAT_SOLVER =
#
ifneq ($(strip $(EXTERNAL_SAT_SOLVER)),)
USE_EXTERNAL_SAT_SOLVER ?= --external-sat-solver $(EXTERNAL_SAT_SOLVER)
endif
CHECKFLAGS += $(USE_EXTERNAL_SAT_SOLVER)
# Job pools
# For version of Litani that are new enough (where `litani print-capabilities`
# prints "pools"), proofs for which `EXPENSIVE = true` is set can be added to a
# "job pool" that restricts how many expensive proofs are run at a time. All
# other proofs will be built in parallel as usual.
#
# In more detail: all compilation, instrumentation, and report jobs are run with
# full parallelism as usual, even for expensive proofs. The CBMC jobs for
# non-expensive proofs are also run in parallel. The only difference is that the
# CBMC safety checks and coverage checks for expensive proofs are run with a
# restricted parallelism level. At any one time, only N of these jobs are run at
# once, amongst all the proofs.
#
# To configure N, Litani needs to be initialized with a pool called "expensive".
# For example, to only run two CBMC safety/coverage jobs at a time from amongst
# all the proofs, you would initialize litani like
# litani init --pools expensive:2
# The run-cbmc-proofs.py script takes care of this initialization through the
# --expensive-jobs-parallelism flag.
#
# To enable this feature, set
# the ENABLE_POOLS variable when running Make, like
# `make ENABLE_POOLS=true report`
# The run-cbmc-proofs.py script takes care of this through the
# --restrict-expensive-jobs flag.
ifeq ($(strip $(ENABLE_POOLS)),)
POOL =
else ifeq ($(strip $(EXPENSIVE)),)
POOL =
else
POOL = --pool expensive
endif
# Similar to the pool feature above. If Litani is new enough, enable
# profiling CBMC's memory use.
ifeq ($(strip $(ENABLE_MEMORY_PROFILING)),)
MEMORY_PROFILING =
else
MEMORY_PROFILING = --profile-memory
endif
# Property checking flags
#
# Each variable below controls a specific property checking flag
# within CBMC. If desired, a property flag can be disabled within
# a particular proof by nulling the corresponding variable. For
# instance, the following line:
#
# CHECK_FLAG_POINTER_CHECK =
#
# would disable the --pointer-check CBMC flag within:
# * an entire project when added to Makefile-project-defines
# * a specific proof when added to the harness Makefile
CBMC_FLAG_MALLOC_MAY_FAIL ?= --malloc-may-fail
CBMC_FLAG_MALLOC_FAIL_NULL ?= --malloc-fail-null
CBMC_FLAG_BOUNDS_CHECK ?= --bounds-check
CBMC_FLAG_CONVERSION_CHECK ?= --conversion-check
CBMC_FLAG_DIV_BY_ZERO_CHECK ?= --div-by-zero-check
CBMC_FLAG_FLOAT_OVERFLOW_CHECK ?= --float-overflow-check
CBMC_FLAG_NAN_CHECK ?= --nan-check
CBMC_FLAG_POINTER_CHECK ?= --pointer-check
CBMC_FLAG_POINTER_OVERFLOW_CHECK ?= --pointer-overflow-check
CBMC_FLAG_POINTER_PRIMITIVE_CHECK ?= --pointer-primitive-check
CBMC_FLAG_SIGNED_OVERFLOW_CHECK ?= --signed-overflow-check
CBMC_FLAG_UNDEFINED_SHIFT_CHECK ?= --undefined-shift-check
CBMC_FLAG_UNSIGNED_OVERFLOW_CHECK ?= --unsigned-overflow-check
CBMC_FLAG_UNWINDING_ASSERTIONS ?= --unwinding-assertions
# CBMC flags used for property checking
CHECKFLAGS += $(CBMC_FLAG_MALLOC_MAY_FAIL)
CHECKFLAGS += $(CBMC_FLAG_MALLOC_FAIL_NULL)
CHECKFLAGS += $(CBMC_FLAG_BOUNDS_CHECK)
CHECKFLAGS += $(CBMC_FLAG_CONVERSION_CHECK)
CHECKFLAGS += $(CBMC_FLAG_DIV_BY_ZERO_CHECK)
CHECKFLAGS += $(CBMC_FLAG_FLOAT_OVERFLOW_CHECK)
CHECKFLAGS += $(CBMC_FLAG_NAN_CHECK)
CHECKFLAGS += $(CBMC_FLAG_POINTER_CHECK)
CHECKFLAGS += $(CBMC_FLAG_POINTER_OVERFLOW_CHECK)
CHECKFLAGS += $(CBMC_FLAG_POINTER_PRIMITIVE_CHECK)
CHECKFLAGS += $(CBMC_FLAG_SIGNED_OVERFLOW_CHECK)
CHECKFLAGS += $(CBMC_FLAG_UNDEFINED_SHIFT_CHECK)
CHECKFLAGS += $(CBMC_FLAG_UNSIGNED_OVERFLOW_CHECK)
CHECKFLAGS += $(CBMC_FLAG_UNWINDING_ASSERTIONS)
# CBMC flags used for coverage checking
COVERFLAGS += $(CBMC_FLAG_MALLOC_MAY_FAIL)
COVERFLAGS += $(CBMC_FLAG_MALLOC_FAIL_NULL)
# Additional CBMC flag to CBMC control verbosity.
#
# Meaningful values are
# 0 none
# 1 only errors
# 2 + warnings
# 4 + results
# 6 + status/phase information
# 8 + statistical information
# 9 + progress information
# 10 + debug info
#
# Uncomment the following line or set in Makefile-project-defines
# CBMC_VERBOSITY ?= --verbosity 4
# Additional CBMC flag to control how CBMC treats static variables.
#
# NONDET_STATIC is a list of flags of the form --nondet-static
# and --nondet-static-exclude VAR. The --nondet-static flag causes
# CBMC to initialize static variables with unconstrained value
# (ignoring initializers and default zero-initialization). The
# --nondet-static-exclude VAR excludes VAR for the variables
# initialized with unconstrained values.
NONDET_STATIC ?= ""
# Flags to pass to goto-cc for compilation and linking
COMPILE_FLAGS ?= -Wall
LINK_FLAGS ?= -Wall
# Preprocessor include paths -I...
INCLUDES ?=
# Preprocessor definitions -D...
DEFINES ?=
# CBMC object model
#
# CBMC_OBJECT_BITS is the number of bits in a pointer CBMC uses for
# the id of the object to which a pointer is pointing. CBMC uses 8
# bits for the object id by default. The remaining bits in the pointer
# are used for offset into the object. This limits the size of the
# objects that CBMC can model. This Makefile defines this bound on
# object size to be CBMC_MAX_OBJECT_SIZE. You are likely to get
# unexpected results if you try to malloc an object larger than this
# bound.
CBMC_OBJECT_BITS ?= 8
# CBMC loop unwinding (Normally set in the proof Makefile)
#
# UNWINDSET is a list of pairs of the form foo.1:4 meaning that
# CBMC should unwind loop 1 in function foo no more than 4 times.
# For historical reasons, the number 4 is one more than the number
# of times CBMC actually unwinds the loop.
UNWINDSET ?=
# CBMC function removal (Normally set set in the proof Makefile)
#
# REMOVE_FUNCTION_BODY is a list of function names. CBMC will "undefine"
# the function, and CBMC will treat the function as having no side effects
# and returning an unconstrained value of the appropriate return type.
# The list should include the names of functions being stubbed out.
REMOVE_FUNCTION_BODY ?=
# CBMC function pointer restriction (Normally set in the proof Makefile)
#
# RESTRICT_FUNCTION_POINTER is a list of function pointer restriction
# instructions of the form:
#
# <fun_id>.function_pointer_call.<N>/<fun_id>[,<fun_id>]*
#
# The function pointer call number <N> in the specified function gets
# rewritten to a case switch over a finite list of functions.
# If some possible target functions are omitted from the list a counter
# example trace will be found by CBMC, i.e. the transformation is sound.
# If the target functions are file-local symbols, then mangled names must
# be used.
RESTRICT_FUNCTION_POINTER ?=
# The project source files (Normally set set in the proof Makefile)
#
# PROJECT_SOURCES is the list of project source files to compile,
# including the source file defining the function under test.
PROJECT_SOURCES ?=
# The proof source files (Normally set in the proof Makefile)
#
# PROOF_SOURCES is the list of proof source files to compile, including
# the proof harness, and including any function stubs being used.
PROOF_SOURCES ?=
# The number of seconds that CBMC should be allowed to run for before
# being forcefully terminated. Currently, this is set to be less than
# the time limit for a CodeBuild job, which is eight hours. If a proof
# run takes longer than the time limit of the CI environment, the
# environment will halt the proof run without updating the Litani
# report, making the proof run appear to "hang".
CBMC_TIMEOUT ?= 21600
# Proof writers could add function contracts in their source code.
# These contracts are ignored by default, but may be enabled in two distinct
# contexts using the following two variables:
# 1. To check whether one or more function contracts are sound with respect to
# the function implementation, CHECK_FUNCTION_CONTRACTS should be a list of
# function names.
# 2. To replace calls to certain functions with their correspondent function
# contracts, USE_FUNCTION_CONTRACTS should be a list of function names.
# One must check separately whether a function contract is sound before
# replacing it in calling contexts.
CHECK_FUNCTION_CONTRACTS ?= ""
CBMC_CHECK_FUNCTION_CONTRACTS := $(patsubst %,--enforce-contract %, $(CHECK_FUNCTION_CONTRACTS))
USE_FUNCTION_CONTRACTS ?= ""
CBMC_USE_FUNCTION_CONTRACTS := $(patsubst %,--replace-call-with-contract %, $(USE_FUNCTION_CONTRACTS))
# Similarly, proof writers could also add loop contracts in their source code
# to obtain unbounded correctness proofs. Unlike function contracts, loop
# contracts are not reusable and thus are checked and used simultaneously.
# These contracts are also ignored by default, but may be enabled by setting
# the APPLY_LOOP_CONTRACTS variable to 1.
APPLY_LOOP_CONTRACTS ?= 0
# Silence makefile output (eg, long litani commands) unless VERBOSE is set.
ifndef VERBOSE
MAKEFLAGS := $(MAKEFLAGS) -s
endif
################################################################
################################################################
## Section II: This section is for project-specific definitions
################################################################
################################################################
## Section II: This section defines the process of running a proof
##
## There should be no reason to edit anything below this line.
################################################################
# Paths
CBMC ?= cbmc
GOTO_ANALYZER ?= goto-analyzer
GOTO_CC ?= goto-cc
GOTO_INSTRUMENT ?= goto-instrument
VIEWER ?= cbmc-viewer
MAKE_SOURCE ?= make-source
VIEWER2 ?= cbmc-viewer
CMAKE ?= cmake
ARPA ?= @echo "You must set ARPA in Makefile-project-defines to run arpa in this project"; false
GOTODIR ?= $(PROOFDIR)/gotos
LOGDIR ?= $(PROOFDIR)/logs
PROJECT ?= project
PROOF ?= proof
HARNESS_GOTO ?= $(GOTODIR)/$(HARNESS_FILE)
PROJECT_GOTO ?= $(GOTODIR)/$(PROJECT)
PROOF_GOTO ?= $(GOTODIR)/$(PROOF)
ARPA_BLDDIR ?= $(abspath $(PROOFDIR)/arpa_cmake)
ARPA_COMP_CMDS ?= $(ARPA_BLDDIR)/compile_commands.json
################################################################
# Useful macros for values that are hard to reference
SPACE :=$() $()
COMMA :=,
################################################################
# Set C compiler defines
CBMCFLAGS += --object-bits $(CBMC_OBJECT_BITS)
DEFINES += -DCBMC=1
DEFINES += -DCBMC_OBJECT_BITS=$(CBMC_OBJECT_BITS)
DEFINES += -DCBMC_MAX_OBJECT_SIZE="(SIZE_MAX>>(CBMC_OBJECT_BITS+1))"
# CI currently assumes cbmc invocation has at most one --unwindset
ifdef UNWINDSET
ifneq ($(strip $(UNWINDSET)),"")
CBMC_UNWINDSET := --unwindset $(subst $(SPACE),$(COMMA),$(strip $(UNWINDSET)))
endif
endif
CBMC_REMOVE_FUNCTION_BODY := $(patsubst %,--remove-function-body %, $(REMOVE_FUNCTION_BODY))
CBMC_RESTRICT_FUNCTION_POINTER := $(patsubst %,--restrict-function-pointer %, $(RESTRICT_FUNCTION_POINTER))
################################################################
# Build targets that make the relevant .goto files
# Compile project sources
$(PROJECT_GOTO)1.goto: $(PROJECT_SOURCES)
$(LITANI) add-job \
--command \
'$(GOTO_CC) $(CBMC_VERBOSITY) --export-file-local-symbols $(COMPILE_FLAGS) $(INCLUDES) $(DEFINES) $^ -o $@' \
--inputs $^ \
--outputs $@ \
--stdout-file $(LOGDIR)/project_sources-log.txt \
--interleave-stdout-stderr \
--pipeline-name "$(PROOF_UID)" \
--ci-stage build \
--description "$(PROOF_UID): building project binary"
# Compile proof sources
$(PROOF_GOTO)1.goto: $(PROOF_SOURCES)
$(LITANI) add-job \
--command \
'$(GOTO_CC) $(CBMC_VERBOSITY) --export-file-local-symbols $(COMPILE_FLAGS) $(INCLUDES) $(DEFINES) $^ -o $@' \
--inputs $^ \
--outputs $@ \
--stdout-file $(LOGDIR)/proof_sources-log.txt \
--interleave-stdout-stderr \
--pipeline-name "$(PROOF_UID)" \
--ci-stage build \
--description "$(PROOF_UID): building proof binary"
# Optionally remove function bodies from project sources
$(PROJECT_GOTO)2.goto: $(PROJECT_GOTO)1.goto
ifeq ($(REMOVE_FUNCTION_BODY),"")
$(LITANI) add-job \
--command 'cp $^ $@' \
--inputs $^ \
--outputs $@ \
--pipeline-name "$(PROOF_UID)" \
--ci-stage build \
--description "$(PROOF_UID): not removing function bodies from project sources"
else
$(LITANI) add-job \
--command \
'$(GOTO_INSTRUMENT) $(CBMC_VERBOSITY) $(CBMC_REMOVE_FUNCTION_BODY) $^ $@' \
--inputs $^ \
--outputs $@ \
--stdout-file $(LOGDIR)/remove_function_body-log.txt \
--interleave-stdout-stderr \
--pipeline-name "$(PROOF_UID)" \
--ci-stage build \
--description "$(PROOF_UID): removing function bodies from project sources"
endif
# Optionally restrict function pointers from project sources
$(PROJECT_GOTO)3.goto: $(PROJECT_GOTO)2.goto
ifeq ($(RESTRICT_FUNCTION_POINTER),"")
$(LITANI) add-job \
--command 'cp $^ $@' \
--inputs $^ \
--outputs $@ \
--pipeline-name "$(PROOF_UID)" \
--ci-stage build \
--description "$(PROOF_UID): not restricting function pointers in project sources"
else
$(LITANI) add-job \
--command \
'$(GOTO_INSTRUMENT) $(CBMC_VERBOSITY) $(CBMC_RESTRICT_FUNCTION_POINTER) $^ $@' \
--inputs $^ \
--outputs $@ \
--stdout-file $(LOGDIR)/restrict_function_pointer-log.txt \
--interleave-stdout-stderr \
--pipeline-name "$(PROOF_UID)" \
--ci-stage build \
--description "$(PROOF_UID): restricting function pointers in project sources"
endif
# Link project and proof sources into the proof harness
$(HARNESS_GOTO)1.goto: $(PROOF_GOTO)1.goto $(PROJECT_GOTO)3.goto
$(LITANI) add-job \
--command '$(GOTO_CC) $(CBMC_VERBOSITY) --function $(HARNESS_ENTRY) $^ $(LINK_FLAGS) -o $@' \
--inputs $^ \
--outputs $@ \
--stdout-file $(LOGDIR)/link_proof_project-log.txt \
--interleave-stdout-stderr \
--pipeline-name "$(PROOF_UID)" \
--ci-stage build \
--description "$(PROOF_UID): linking project to proof"
# Optionally check function contracts
$(HARNESS_GOTO)2.goto: $(HARNESS_GOTO)1.goto
ifeq ($(CHECK_FUNCTION_CONTRACTS),"")
$(LITANI) add-job \
--command 'cp $^ $@' \
--inputs $^ \
--outputs $@ \
--pipeline-name "$(PROOF_UID)" \
--ci-stage build \
--description "$(PROOF_UID): not checking function contracts"
else
$(LITANI) add-job \
--command \
'$(GOTO_INSTRUMENT) $(CBMC_VERBOSITY) $(CBMC_CHECK_FUNCTION_CONTRACTS) $^ $@' \
--inputs $^ \
--outputs $@ \
--stdout-file $(LOGDIR)/check_function_contracts-log.txt \
--interleave-stdout-stderr \
--pipeline-name "$(PROOF_UID)" \
--ci-stage build \
--description "$(PROOF_UID): checking function contracts"
endif
# Optionally replace function calls with function contracts
$(HARNESS_GOTO)3.goto: $(HARNESS_GOTO)2.goto
ifeq ($(USE_FUNCTION_CONTRACTS),"")
$(LITANI) add-job \
--command 'cp $^ $@' \
--inputs $^ \
--outputs $@ \
--pipeline-name "$(PROOF_UID)" \
--ci-stage build \
--description "$(PROOF_UID): not replacing function calls with function contracts"
else
$(LITANI) add-job \
--command \
'$(GOTO_INSTRUMENT) $(CBMC_VERBOSITY) $(CBMC_USE_FUNCTION_CONTRACTS) $^ $@' \
--inputs $^ \
--outputs $@ \
--stdout-file $(LOGDIR)/use_function_contracts-log.txt \
--interleave-stdout-stderr \
--pipeline-name "$(PROOF_UID)" \
--ci-stage build \
--description "$(PROOF_UID): replacing function calls with function contracts"
endif
# Optionally apply loop contracts
$(HARNESS_GOTO)4.goto: $(HARNESS_GOTO)3.goto
ifneq ($(APPLY_LOOP_CONTRACTS),1)
$(LITANI) add-job \
--command 'cp $^ $@' \
--inputs $^ \
--outputs $@ \
--pipeline-name "$(PROOF_UID)" \
--ci-stage build \
--description "$(PROOF_UID): not applying loop contracts"
else
$(LITANI) add-job \
--command \
'$(GOTO_INSTRUMENT) $(CBMC_VERBOSITY) --apply-loop-contracts $^ $@' \
--inputs $^ \
--outputs $@ \
--stdout-file $(LOGDIR)/apply_loop_contracts-log.txt \
--interleave-stdout-stderr \
--pipeline-name "$(PROOF_UID)" \
--ci-stage build \
--description "$(PROOF_UID): applying loop contracts"
endif
# Optionally fill static variable with unconstrained values
$(HARNESS_GOTO)5.goto: $(HARNESS_GOTO)4.goto
ifeq ($(NONDET_STATIC),"")
$(LITANI) add-job \
--command 'cp $^ $@' \
--inputs $^ \
--outputs $@ \
--pipeline-name "$(PROOF_UID)" \
--ci-stage build \
--description "$(PROOF_UID): not setting static variables to nondet"
else
$(LITANI) add-job \
--command \
'$(GOTO_INSTRUMENT) $(CBMC_VERBOSITY) $(NONDET_STATIC) $^ $@' \
--inputs $^ \
--outputs $@ \
--stdout-file $(LOGDIR)/nondet_static-log.txt \
--interleave-stdout-stderr \
--pipeline-name "$(PROOF_UID)" \
--ci-stage build \
--description "$(PROOF_UID): setting static variables to nondet"
endif
# Omit unused functions (sharpens coverage calculations)
$(HARNESS_GOTO)6.goto: $(HARNESS_GOTO)5.goto
$(LITANI) add-job \
--command \
'$(GOTO_INSTRUMENT) $(CBMC_VERBOSITY) --drop-unused-functions $^ $@' \
--inputs $^ \
--outputs $@ \
--stdout-file $(LOGDIR)/drop_unused_functions-log.txt \
--interleave-stdout-stderr \
--pipeline-name "$(PROOF_UID)" \
--ci-stage build \
--description "$(PROOF_UID): dropping unused functions"
# Omit initialization of unused global variables (reduces problem size)
$(HARNESS_GOTO)7.goto: $(HARNESS_GOTO)6.goto
$(LITANI) add-job \
--command \
'$(GOTO_INSTRUMENT) $(CBMC_VERBOSITY) --slice-global-inits $^ $@' \
--inputs $^ \
--outputs $@ \
--stdout-file $(LOGDIR)/slice_global_inits-log.txt \
--interleave-stdout-stderr \
--pipeline-name "$(PROOF_UID)" \
--ci-stage build \
--description "$(PROOF_UID): slicing global initializations"
# Final name for proof harness
$(HARNESS_GOTO).goto: $(HARNESS_GOTO)7.goto
$(LITANI) add-job \
--command 'cp $< $@' \
--inputs $^ \
--outputs $@ \
--pipeline-name "$(PROOF_UID)" \
--ci-stage build \
--description "$(PROOF_UID): copying final goto-binary"
################################################################
# Targets to run Arpa
$(ARPA_BLDDIR):
$(CMAKE) $(ARPA_CMAKE_FLAGS) \
-DCMAKE_EXPORT_COMPILE_COMMANDS=1 \
-B $(ARPA_BLDDIR) \
-S $(SRCDIR)
arpa: $(ARPA_BLDDIR)
$(ARPA) run -cc $(ARPA_COMP_CMDS) -r $(SRCDIR)
################################################################
# Targets to run the analysis commands
$(LOGDIR)/result.txt: $(HARNESS_GOTO).goto
$(LITANI) add-job \
$(POOL) \
--command \
'$(CBMC) $(CBMC_VERBOSITY) $(CBMCFLAGS) $(CHECKFLAGS) --trace $<' \
--inputs $^ \
--outputs $@ \
--ci-stage test \
--stdout-file $@ \
$(MEMORY_PROFILING) \
--ignore-returns 10 \
--timeout $(CBMC_TIMEOUT) \
--pipeline-name "$(PROOF_UID)" \
--tags "stats-group:safety checks" \
--stderr-file $(LOGDIR)/result-err-log.txt \
--description "$(PROOF_UID): checking safety properties"
$(LOGDIR)/result.xml: $(HARNESS_GOTO).goto
$(LITANI) add-job \
$(POOL) \
--command \
'$(CBMC) $(CBMC_VERBOSITY) $(CBMCFLAGS) $(CHECKFLAGS) --trace --xml-ui $<' \
--inputs $^ \
--outputs $@ \
--ci-stage test \
--stdout-file $@ \
$(MEMORY_PROFILING) \
--ignore-returns 10 \
--timeout $(CBMC_TIMEOUT) \
--pipeline-name "$(PROOF_UID)" \
--tags "stats-group:safety checks" \
--stderr-file $(LOGDIR)/result-err-log.txt \
--description "$(PROOF_UID): checking safety properties"
$(LOGDIR)/property.xml: $(HARNESS_GOTO).goto
$(LITANI) add-job \
--command \
'$(CBMC) $(CBMC_VERBOSITY) $(CBMCFLAGS) $(CHECKFLAGS) --show-properties --xml-ui $<' \
--inputs $^ \
--outputs $@ \
--ci-stage test \
--stdout-file $@ \
--ignore-returns 10 \
--pipeline-name "$(PROOF_UID)" \
--stderr-file $(LOGDIR)/property-err-log.txt \
--description "$(PROOF_UID): printing safety properties"
$(LOGDIR)/coverage.xml: $(HARNESS_GOTO).goto
$(LITANI) add-job \
$(POOL) \
--command \
'$(CBMC) $(CBMC_VERBOSITY) $(CBMCFLAGS) $(COVERFLAGS) --cover location --xml-ui $<' \
--inputs $^ \
--outputs $@ \
--ci-stage test \
--stdout-file $@ \
$(MEMORY_PROFILING) \
--ignore-returns 10 \
--timeout $(CBMC_TIMEOUT) \
--pipeline-name "$(PROOF_UID)" \
--tags "stats-group:coverage computation" \
--stderr-file $(LOGDIR)/coverage-err-log.txt \
--description "$(PROOF_UID): calculating coverage"
define VIEWER_CMD
$(VIEWER) \
--result $(LOGDIR)/result.txt \
--block $(LOGDIR)/coverage.xml \
--property $(LOGDIR)/property.xml \
--srcdir $(SRCDIR) \
--goto $(HARNESS_GOTO).goto \
--htmldir $(PROOFDIR)/html
endef
export VIEWER_CMD
$(PROOFDIR)/html: $(LOGDIR)/result.txt $(LOGDIR)/property.xml $(LOGDIR)/coverage.xml
$(LITANI) add-job \
--command "$$VIEWER_CMD" \
--inputs $^ \
--outputs $(PROOFDIR)/html \
--pipeline-name "$(PROOF_UID)" \
--ci-stage report \
--stdout-file $(LOGDIR)/viewer-log.txt \
--interleave-stdout-stderr \
--description "$(PROOF_UID): generating report"
# Caution: run make-source before running property and coverage checking
# The current make-source script removes the goto binary
$(LOGDIR)/source.json:
mkdir -p $(dir $@)
$(RM) -r $(GOTODIR)
$(MAKE_SOURCE) --srcdir $(SRCDIR) --wkdir $(PROOFDIR) > $@
$(RM) -r $(GOTODIR)
define VIEWER2_CMD
$(VIEWER2) \
--result $(LOGDIR)/result.xml \
--coverage $(LOGDIR)/coverage.xml \
--property $(LOGDIR)/property.xml \
--srcdir $(SRCDIR) \
--goto $(HARNESS_GOTO).goto \
--reportdir $(PROOFDIR)/report
endef
export VIEWER2_CMD
# Omit logs/source.json from report generation until make-sources
# works correctly with Makefiles that invoke the compiler with
# mutliple source files at once.
$(PROOFDIR)/report: $(LOGDIR)/result.xml $(LOGDIR)/property.xml $(LOGDIR)/coverage.xml
$(LITANI) add-job \
--command "$$VIEWER2_CMD" \
--inputs $^ \
--outputs $(PROOFDIR)/report \
--pipeline-name "$(PROOF_UID)" \
--stdout-file $(LOGDIR)/viewer-log.txt \
--interleave-stdout-stderr \
--ci-stage report \
--description "$(PROOF_UID): generating report"
litani-path:
@echo $(LITANI)
# ##############################################################
# Phony Rules
#
# These rules provide a convenient way to run a single proof up to a
# certain stage. Users can browse into a proof directory and run
# "make -Bj 3 report" to generate a report for just that proof, or
# "make goto" to build the goto binary. Under the hood, this runs litani
# for just that proof.
_goto: $(HARNESS_GOTO).goto
goto:
@ echo Running 'litani init'
$(LITANI) init --project $(PROJECT_NAME)
@ echo Running 'litani add-job'
$(MAKE) -B _goto
@ echo Running 'litani build'
$(LITANI) run-build
_result: $(LOGDIR)/result.txt
result:
@ echo Running 'litani init'
$(LITANI) init --project $(PROJECT_NAME)
@ echo Running 'litani add-job'
$(MAKE) -B _result
@ echo Running 'litani build'
$(LITANI) run-build
_property: $(LOGDIR)/property.xml
property:
@ echo Running 'litani init'
$(LITANI) init --project $(PROJECT_NAME)
@ echo Running 'litani add-job'
$(MAKE) -B _property
@ echo Running 'litani build'
$(LITANI) run-build
_coverage: $(LOGDIR)/coverage.xml
coverage:
@ echo Running 'litani init'
$(LITANI) init --project $(PROJECT_NAME)
@ echo Running 'litani add-job'
$(MAKE) -B _coverage
@ echo Running 'litani build'
$(LITANI) run-build
# Choose the invocation of cbmc-viewer depending on which version of
# cbmc-viewer is installed. The --version flag is not implemented in
# version 1 --- it is an "unrecognized argument" --- but it is
# implemented in version 2.
_report1: $(PROOFDIR)/html
_report2: $(PROOFDIR)/report
_report:
(cbmc-viewer --version 2>&1 | grep "unrecognized argument" > /dev/null) && \
$(MAKE) -B _report1 || $(MAKE) -B _report2
report report1 report2:
@ echo Running 'litani init'
$(LITANI) init --project $(PROJECT_NAME)
@ echo Running 'litani add-job'
$(MAKE) -B _report
@ echo Running 'litani build'
$(LITANI) run-build
################################################################
# Targets to clean up after ourselves
clean:
-$(RM) $(DEPENDENT_GOTOS)
-$(RM) TAGS*
-$(RM) *~ \#*
-$(RM) Makefile.arpa
-$(RM) -r $(ARPA_BLDDIR)
veryclean: clean
-$(RM) -r html report
-$(RM) -r $(LOGDIR) $(GOTODIR)
.PHONY: \
_coverage \
_goto \
_property \
_report \
_report2 \
_result \
arpa \
clean \
coverage \
goto \
litani-path \
property \
report \
report2 \
result \
setup_dependencies \
testdeps \
veryclean \
#
################################################################
# Rule for generating cbmc-batch.yaml, used by the CI at
# https://github.com/awslabs/aws-batch-cbmc/
JOB_OS ?= ubuntu16
JOB_MEMORY ?= 32000
# Proofs that are expected to fail should set EXPECTED to
# "FAILED" in their Makefile. Values other than SUCCESSFUL
# or FAILED will cause a CI error.
EXPECTED ?= SUCCESSFUL
define yaml_encode_options
"$(shell echo $(1) | sed 's/ ,/ /g' | sed 's/ /;/g')"
endef
CI_FLAGS = $(CBMCFLAGS) $(CHECKFLAGS) $(COVERFLAGS)
cbmc-batch.yaml:
@$(RM) $@
@echo 'build_memory: $(JOB_MEMORY)' > $@
@echo 'cbmcflags: $(strip $(call yaml_encode_options,$(CI_FLAGS)))' >> $@
@echo 'coverage_memory: $(JOB_MEMORY)' >> $@
@echo 'expected: $(EXPECTED)' >> $@
@echo 'goto: $(HARNESS_GOTO).goto' >> $@
@echo 'jobos: $(JOB_OS)' >> $@
@echo 'property_memory: $(JOB_MEMORY)' >> $@
@echo 'report_memory: $(JOB_MEMORY)' >> $@
.PHONY: cbmc-batch.yaml
################################################################
# Run "make echo-proof-uid" to print the proof ID of a proof. This can be
# used by scripts to ensure that every proof has an ID, that there are
# no duplicates, etc.
.PHONY: echo-proof-uid
echo-proof-uid:
@echo $(PROOF_UID)
.PHONY: echo-project-name
echo-project-name:
@echo $(PROJECT_NAME)
################################################################
# Project-specific targets requiring values defined above
sinclude $(PROOF_ROOT)/Makefile-project-targets
# CI-specific targets to drive cbmc in CI
sinclude $(PROOF_ROOT)/Makefile-project-testing
################################################################

27
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/proofs/README.md Normal file
View File

@@ -0,0 +1,27 @@
CBMC proofs
===========
This directory contains the CBMC proofs. Each proof is in its own
directory.
This directory includes four Makefiles.
One Makefile describes the basic workflow for building and running proofs:
* Makefile.common:
* make: builds the goto binary, does the cbmc property checking
and coverage checking, and builds the final report.
* make goto: builds the goto binary
* make result: does cbmc property checking
* make coverage: does cbmc coverage checking
* make report: builds the final report
Three included Makefiles describe project-specific settings and can override
definitions in Makefile.common:
* Makefile-project-defines: definitions like compiler flags
required to build the goto binaries, and definitions to override
definitions in Makefile.common.
* Makefile-project-targets: other make targets needed for the project
* Makefile-project-testing: other definitions and targets needed for
unit testing or continuous integration.

368
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/proofs/run-cbmc-proofs.py Executable file
View File

@@ -0,0 +1,368 @@
#!/usr/bin/env python3
#
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0
import argparse
import asyncio
import json
import logging
import math
import os
import pathlib
import re
import subprocess
import sys
DESCRIPTION = "Configure and run all CBMC proofs in parallel"
# Keep the epilog hard-wrapped at 70 characters, as it gets printed
# verbatim in the terminal. 70 characters stops here --------------> |
EPILOG = """
This tool automates the process of running `make report` in each of
the CBMC proof directories. The tool calculates the dependency graph
of all tasks needed to build, run, and report on all the proofs, and
executes these tasks in parallel.
The tool is roughly equivalent to doing this:
litani init --project "my-cool-project";
find . -name cbmc-proof.txt | while read -r proof; do
pushd $(dirname ${proof});
# The `make _report` rule adds a single proof to litani
# without running it
make _report;
popd;
done
litani run-build;
except that it is much faster and provides some convenience options.
The CBMC CI runs this script with no arguments to build and run all
proofs in parallel. The value of "my-cool-project" is taken from the
PROJECT_NAME variable in Makefile-project-defines.
The --no-standalone argument omits the `litani init` and `litani
run-build`; use it when you want to add additional proof jobs, not
just the CBMC ones. In that case, you would run `litani init`
yourself; then run `run-cbmc-proofs --no-standalone`; add any
additional jobs that you want to execute with `litani add-job`; and
finally run `litani run-build`.
The litani dashboard will be written under the `output` directory; the
cbmc-viewer reports remain in the `$PROOF_DIR/report` directory. The
HTML dashboard from the latest Litani run will always be symlinked to
`output/latest/html/index.html`, so you can keep that page open in
your browser and reload the page whenever you re-run this script.
"""
# 70 characters stops here ----------------------------------------> |
def get_project_name():
cmd = [
"make",
"-f", "Makefile.common",
"echo-project-name",
]
logging.debug(" ".join(cmd))
proc = subprocess.run(cmd, universal_newlines=True, stdout=subprocess.PIPE)
if proc.returncode:
logging.critical("could not run make to determine project name")
sys.exit(1)
if not proc.stdout.strip():
logging.warning(
"project name has not been set; using generic name instead. "
"Set the PROJECT_NAME value in Makefile-project-defines to "
"remove this warning")
return "<PROJECT NAME HERE>"
return proc.stdout.strip()
def get_args():
pars = argparse.ArgumentParser(
description=DESCRIPTION, epilog=EPILOG,
formatter_class=argparse.RawDescriptionHelpFormatter)
for arg in [{
"flags": ["-j", "--parallel-jobs"],
"type": int,
"metavar": "N",
"help": "run at most N proof jobs in parallel",
}, {
"flags": ["--no-standalone"],
"action": "store_true",
"help": "only configure proofs: do not initialize nor run",
}, {
"flags": ["-p", "--proofs"],
"nargs": "+",
"metavar": "DIR",
"help": "only run proof in directory DIR (can pass more than one)",
}, {
"flags": ["--project-name"],
"metavar": "NAME",
"default": get_project_name(),
"help": "project name for report. Default: %(default)s",
}, {
"flags": ["--marker-file"],
"metavar": "FILE",
"default": "cbmc-proof.txt",
"help": (
"name of file that marks proof directories. Default: "
"%(default)s"),
}, {
"flags": ["--no-memory-profile"],
"action": "store_true",
"help": "disable memory profiling, even if Litani supports it"
}, {
"flags": ["--no-expensive-limit"],
"action": "store_true",
"help": "do not limit parallelism of 'EXPENSIVE' jobs",
}, {
"flags": ["--expensive-jobs-parallelism"],
"metavar": "N",
"default": 1,
"type": int,
"help": (
"how many proof jobs marked 'EXPENSIVE' to run in parallel. "
"Default: %(default)s"),
}, {
"flags": ["--verbose"],
"action": "store_true",
"help": "verbose output",
}]:
flags = arg.pop("flags")
pars.add_argument(*flags, **arg)
return pars.parse_args()
def set_up_logging(verbose):
if verbose:
level = logging.DEBUG
else:
level = logging.WARNING
logging.basicConfig(
format="run-cbmc-proofs: %(message)s", level=level)
def task_pool_size():
ret = os.cpu_count()
if ret is None or ret < 3:
return 1
return ret - 2
def print_counter(counter):
print(
"\rConfiguring CBMC proofs: "
"{complete:{width}} / {total:{width}}".format(
**counter), end="", file=sys.stderr)
def get_proof_dirs(proof_root, proof_list, marker_file):
if proof_list is not None:
proofs_remaining = list(proof_list)
else:
proofs_remaining = []
for root, _, fyles in os.walk(proof_root):
proof_name = str(pathlib.Path(root).name)
if proof_list and proof_name not in proof_list:
continue
if proof_list and proof_name in proofs_remaining:
proofs_remaining.remove(proof_name)
if marker_file in fyles:
yield root
if proofs_remaining:
logging.critical(
"The following proofs were not found: %s",
", ".join(proofs_remaining))
sys.exit(1)
def run_build(litani, jobs):
cmd = [str(litani), "run-build"]
if jobs:
cmd.extend(["-j", str(jobs)])
logging.debug(" ".join(cmd))
proc = subprocess.run(cmd)
if proc.returncode:
logging.critical("Failed to run litani run-build")
sys.exit(1)
def get_litani_path(proof_root):
cmd = [
"make",
"PROOF_ROOT=%s" % proof_root,
"-f", "Makefile.common",
"litani-path",
]
logging.debug(" ".join(cmd))
proc = subprocess.run(cmd, universal_newlines=True, stdout=subprocess.PIPE)
if proc.returncode:
logging.critical("Could not determine path to litani")
sys.exit(1)
return proc.stdout.strip()
def get_litani_capabilities(litani_path):
cmd = [litani_path, "print-capabilities"]
proc = subprocess.run(
cmd, text=True, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL)
if proc.returncode:
return []
try:
return json.loads(proc.stdout)
except RuntimeError:
logging.warning("Could not load litani capabilities: '%s'", proc.stdout)
return []
def check_uid_uniqueness(proof_dir, proof_uids):
with (pathlib.Path(proof_dir) / "Makefile").open() as handle:
for line in handle:
m = re.match(r"^PROOF_UID\s*=\s*(?P<uid>\w+)", line)
if not m:
continue
if m["uid"] not in proof_uids:
proof_uids[m["uid"]] = proof_dir
return
logging.critical(
"The Makefile in directory '%s' should have a different "
"PROOF_UID than the Makefile in directory '%s'",
proof_dir, proof_uids[m["uid"]])
sys.exit(1)
logging.critical(
"The Makefile in directory '%s' should contain a line like", proof_dir)
logging.critical("PROOF_UID = ...")
logging.critical("with a unique identifier for the proof.")
sys.exit(1)
def should_enable_memory_profiling(litani_caps, args):
if args.no_memory_profile:
return False
return "memory_profile" in litani_caps
def should_enable_pools(litani_caps, args):
if args.no_expensive_limit:
return False
return "pools" in litani_caps
async def configure_proof_dirs(
queue, counter, proof_uids, enable_pools, enable_memory_profiling):
while True:
print_counter(counter)
path = str(await queue.get())
check_uid_uniqueness(path, proof_uids)
pools = ["ENABLE_POOLS=true"] if enable_pools else []
profiling = [
"ENABLE_MEMORY_PROFILING=true"] if enable_memory_profiling else []
proc = await asyncio.create_subprocess_exec(
# Allow interactive tasks to preempt proof configuration
"nice", "-n", "15", "make", *pools, *profiling, "-B", "--quiet",
"_report", cwd=path)
await proc.wait()
counter["fail" if proc.returncode else "pass"].append(path)
counter["complete"] += 1
print_counter(counter)
queue.task_done()
async def main():
args = get_args()
set_up_logging(args.verbose)
proof_root = pathlib.Path(os.getcwd())
litani = get_litani_path(proof_root)
litani_caps = get_litani_capabilities(litani)
enable_pools = should_enable_pools(litani_caps, args)
init_pools = [
"--pools", f"expensive:{args.expensive_jobs_parallelism}"
] if enable_pools else []
if not args.no_standalone:
cmd = [
str(litani), "init", *init_pools, "--project", args.project_name,
"--no-print-out-dir",
]
if "output_directory_flags" in litani_caps:
out_prefix = proof_root / "output"
out_symlink = out_prefix / "latest"
out_index = out_symlink / "html" / "index.html"
cmd.extend([
"--output-prefix", str(out_prefix),
"--output-symlink", str(out_symlink),
])
print(
"\nFor your convenience, the output of this run will be "
"symbolically linked to %s\n" % str(out_index))
logging.debug(" ".join(cmd))
proc = subprocess.run(cmd)
if proc.returncode:
logging.critical("Failed to run litani init")
sys.exit(1)
proof_dirs = list(get_proof_dirs(
proof_root, args.proofs, args.marker_file))
if not proof_dirs:
logging.critical("No proof directories found")
sys.exit(1)
proof_queue = asyncio.Queue()
for proof_dir in proof_dirs:
proof_queue.put_nowait(proof_dir)
counter = {
"pass": [],
"fail": [],
"complete": 0,
"total": len(proof_dirs),
"width": int(math.log10(len(proof_dirs))) + 1
}
proof_uids = {}
tasks = []
enable_memory_profiling = should_enable_memory_profiling(litani_caps, args)
for _ in range(task_pool_size()):
task = asyncio.create_task(configure_proof_dirs(
proof_queue, counter, proof_uids, enable_pools,
enable_memory_profiling))
tasks.append(task)
await proof_queue.join()
print_counter(counter)
print("", file=sys.stderr)
if counter["fail"]:
logging.critical(
"Failed to configure the following proofs:\n%s", "\n".join(
[str(f) for f in counter["fail"]]))
sys.exit(1)
if not args.no_standalone:
run_build(litani, args.parallel_jobs)
if __name__ == "__main__":
asyncio.run(main())

6
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/sources/README.md Normal file
View File

@@ -0,0 +1,6 @@
CBMC proof source code
======================
This directory contains source code written for CBMC proofs. It is
common to write some code to model aspects of the system under test,
and this code goes here.

6
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/template-for-repository/stubs/README.md Normal file
View File

@@ -0,0 +1,6 @@
CBMC proof stubs
======================
This directory contains the stubs written for CBMC proofs. It is
common to stub out functionality like network send and receive methods
when writing a CBMC proof, and the code for these stubs goes here.

40
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/training-material/CODING-FOR-VERIFICATION.md Normal file
View File

@@ -0,0 +1,40 @@
# CBMC Coding Guidelines
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
**Table of Contents**
- [Code Organization to Support Verification](#code-organization-to-support-verification)
- [Improving Verification Performance](#improving-verification-performance)
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
The basic principles of coding for verification are similar to those of coding for testability, with some modifications due to the nature of the SAT solver underlying CBMC.
## Code Organization to Support Verification
* Write small functions. Functions should:
* Do exactly one thing.
* Take all their input and outputs through function parameters and rely as little as possible on global state. Where possible, avoid global variables.
* Encapsulate interaction with the environment within a small function. Interaction with the environment includes accessing files, the network, etc. This makes is possible to verify that function independently and then stub it out for the rest of the verification.
* Functions should check their input parameters, and return an error code when they fail to verify.
This makes harnesses much simpler, since any value for the parameters is a valid input to the function.
* Avoid unbounded loops as far as possible, and encapsulate the ones that you need. CBMC does bounded model checking, so we need to be able to compute a bound on the number of iterations of any given loop. Loops that iterate a constant number of times are best. Loops whose iteration depends on input will require making some assumptions about the input.
* Consider defining magic numbers that control loop bounds and buffer sizes in your build system, i.e., `-DBUFFER_SIZE=1024` and similar. This ensures that you can configure this value at build time, and we can also use those values in our proofs.
* Provide an easy way to access static functions and data structures for testing, if you must have them. For example, use a macro that overrides static.
* Make threads independently verifiable. When writing concurrent programs, reduce interaction to well-defined points. This enables verification of each thread in isolation.
## Improving Verification Performance
* Avoid void pointers (`void*`). There are two reasons people use void pointers:
* To hide implementation detail. This use of void pointers is unnecessary, because we can replace `void *bar` with `struct foo *bar` and declare `struct foo` later within the implementation.
* To implement a form of polymorphism. Don't do this for gratuitous reasons (e.g., because it might someday be useful). Void pointers can block constant propagation which can dramatically reduce the size of the formula constructed for the constraint solver.
* Avoid function pointers. When unavoidable, ensure that function pointer types have a unique signature. They really can make the difference between a proof and no proof. When CBMC encounters a function pointer, it has to consider all possibilities for what that function could be, based on loose signature matching. CBMC has to consider possible any function in the entire program whose address is taken with a signature matching the function pointer. So for each function invocation, the symbolic execution of a single function is replaced with the symbolic execution of a collection of functions (including the functions they call), and the combinatorial explosion makes the size of the formula too big for memory. The worst thing you can do is to give your functions the signature `void foo(void *arg)`; see the point above about avoiding `void*`.
* Large (more than several kB in size) arrays can cause trouble. Again, defining the sizes of arrays in the build system means that we can cleanly re-define them to smaller bounds for our proofs.
* Data-structures should explicitly carry their size, as a parameter (e.g., Pascal strings are better than C strings).
* Stay type safe.
* Allocate the correct size of objects. Don't use smaller structs when you're only using some fields.
* Consider encapsulating loops in a function, or even just the loop body. Nested loops can lead to a combinatorial explosion in the size of the formula sent to the constraint solver. Encapsulated loops can be specified and validated in isolation, and the simpler specification can be used in place of the function in the rest of the validation.
* Try to minimize string comparisons
* E.g., instead of making a `string->string` hash table, consider an `enum->string` hash-table.

198
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/training-material/DEBUG-CBMC.md Normal file
View File

@@ -0,0 +1,198 @@
# Debugging CBMC issues
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
**Table of Contents**
- [I see 12 proof failures. How do I select which one to debug?](#i-see-12-proof-failures-how-do-i-select-which-one-to-debug)
- [How do I debug a proof failure?](#how-do-i-debug-a-proof-failure)
- [Read the trace](#read-the-trace)
- [Add additional information to the trace](#add-additional-information-to-the-trace)
- [Delta debugging](#delta-debugging)
- [Add assertions to check your hypotheses.](#add-assertions-to-check-your-hypotheses)
- [Use `assert(0)` to dump program state leading to a checkpoint](#use-assert0-to-dump-program-state-leading-to-a-checkpoint)
- [Use `assume(...)` to block uninteresting paths](#use-assume-to-block-uninteresting-paths)
- [Consider the possibility it is a fault in the code itself](#consider-the-possibility-it-is-a-fault-in-the-code-itself)
- [How do I improve proofs with low coverage?](#how-do-i-improve-proofs-with-low-coverage)
- [Fix any CBMC errors](#fix-any-cbmc-errors)
- [Check for truly unreachable code.](#check-for-truly-unreachable-code)
- [Check for over-constrained inputs](#check-for-over-constrained-inputs)
- [How can I tell if my proof is over-constrained?](#how-can-i-tell-if-my-proof-is-over-constrained)
- [What should I do if CBMC crashes?](#what-should-i-do-if-cbmc-crashes)
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
## I see 12 proof failures. How do I select which one to debug?
CBMC proof failures seem to come in batches: you run the proof, and see a dozen different errors reported
In many cases, these failures are related: instead of stressing about the number of failures, pick one, debug it, and see if fixing it removes (many of) the others.
Some good heuristics for deciding which failure to investigate:
1. **Look for a failure that occurs early on in the proof.**
This will often be the one with the shortest trace [TODO viewer should output this information].
The shorter the trace leading to the issue, the easier it is to debug.
1. **Look for a failure in code you understand.**
Some functions are simpler than others: a failure in a simple function is often easier to analyze that one in a complicated function.
And a failure in a function you understand is easier than one in a function you are not familiar with.
1. **Look for a simple type of failure.**
For example, the trace from a null dereference is often easier to follow than the trace for a use of a DEAD pointer.
But they're normally exactly the same bug!
Since null dereference bugs normally give the simplest traces, start with them first.
Often, resolving the null dereference also fixes the other related bugs.
## How do I debug a proof failure?
There are a number of techniques that have proven useful in debugging proof failures.
### Read the trace
[TODO link to a guide to viewer]
CBMC viewer generates a step-by-step trace that leads to the assertion violation.
This trace details
* Every line of code executed
* Every function call made
* Every time a variable is assigned
Essentially, this trace contains everything you would get from attaching a debugger to the program, and single stepping until the violation occurred.
Take a look at the values of the relevant variables right before the assertion violation.
Do they make sense?
If not, figure out where they were assigned.
I often find that `Ctrl-F` is my friend here: I search for either the variable name, or the value it was assigned, and see where it appears in the trace.
Similarly, look at the set of function calls that led to the error.
Do they make sense?
Are there functions you expect to see there, but don't?
Are there functions you didn't expect to see there, but do?
### Add additional information to the trace
The trace has all the information you need to understand the state of program memory at every point during the execution.
But its not always that easy to reconstruct.
In particular, the trace records the value of a variable when it is written to.
But it doesn't record the value of a variable that is only read, or passed along to another function.
You can solve this by adding "dummy writes" to the program.
For example, let's say you were debugging an error that involved the following function
```
int foo(struct bar* b, int x) {
baz(b->data, x);
}
```
Figuring out the value of `b->data` and `x` are possible given a complete trace, but its difficult.
Any it might harder to figure out the value of `b->size`.
Instead, annotate the code to track those values:
```
int foo(struct bar* b, int x) {
struct bar debug_foo_b = *b;
int debug_foo_x = x;
baz(b->data, x);
}
```
the trace will now contain an assignment to `debug_foo_b`, which will let you see what values each member of the struct had.
### Delta debugging
[Delta debugging](http://web2.cs.columbia.edu/~junfeng/09fa-e6998/papers/delta-debug.pdf) is a powerful technique for localizing faults and creating minimal reproducing test-cases.
Essentially, you modify the program in some way, typically either by removing (commenting out) or modifying code.
You then rerun the verification tool, and see if the results changed.
The goal is to either:
1. produce a small program which still displays the bug or
1. produce a small change between two programs, one of which has the bug, and the other doesn't.
In case 1, you now have a small program which is hopefully easy to understand;
In case 2, you have a small change which induces the bug, and hopefully leads you toward the root cause.
### Add assertions to check your hypotheses.
For example, consider the case of a null pointer dereference of a pointer `p`.
It is important to distinguish the case where the pointer *must* be null, vs the case where it *may* be null, vs the case where it *is never* null.
You can test for these cases by adding `assert(p)` to the function.
If the can be null, the assertion will trigger.
If it cannot be null, the assertion will succeed.
Now, check `assert(!p)` instead.
If can be non-null, this assertion will fail.
If it can only be null, this assertion will succeed.
You now know which one of the three cases is true.
And you can use the trace to see why it can be null/non-null.
You can do similar things to determine why a branch is reachable, or unreachable.
### Use `assert(0)` to dump program state leading to a checkpoint
Sometimes, you want to know how/whether a particular line of code is reachable.
One easy way to learn that is to put `assert(0)` right before the line.
CBMC will detect the assertion violation, and give a trace explaining how it reached there, and with what values.
If the assertion passes without error, you know that the line is unreachable given the current proof harness.
### Use `assume(...)` to block uninteresting paths
There are often many possible execution paths that reach a given line of code / assertion.
Some of these may reflect cases you are trying to understand, while others do not help with your current debugging plan.
Left to its own devices, CBMC will non-deterministically choose one of those traces, which may not be the one you want.
You can guide CBMC to the trace you want by sprinkling `__CPROVER_assume()` statements within the code.
For example, you might `__CPROVER_assume()` that a function fails with an error code, to test whether the calling function handles that error code correctly.
Or you might `__CPROVER_assume()` that a given variable is null, to simplify you search for the root cause of a null dereference.
### Consider the possibility it is a fault in the code itself
In many cases, the error detected by CBMC represents a true issue within the code itself.
This is particularly common in the case of functions which fail to validate their inputs.
In this case, the fix is either to validate the inputs, and return an error if given invalid inputs, or to document the requirements on the inputs, and state that actions on illegal inputs are undefined behaviour.
Which solution you choose depends on the risk profile of the code.
It is also common that code being verified has integer-overflows and other errors that only occur in unusual circumstances.
In these cases, the solution is to either guarantee that inputs are sufficiently small to prevent these issues, or to use overflow-safe builtins, such as gcc's `__builtin_mul_overflow` (documented [here](https://gcc.gnu.org/onlinedocs/gcc/Integer-Overflow-Builtins.html)).
## How do I improve proofs with low coverage?
### Fix any CBMC errors
Make sure that there are no missing function definitions, or property violations.
Both of these errors can affect coverage calculations.
### Check for truly unreachable code.
In some cases, code may be truly unreachable - for example, redundant defensive checks.
Or this may be code which is sometimes reachable, but not in the context of your proof.
For example:
```
int size_from_enum(type_enum t) {
switch (t) {
case BAR: return 1;
case BAZ: return 2;
...
}
int function_being_tested() {
return size_from_enum(BAZ);
}
```
In this case, most of the lines in `size_from_enum` will appear to be unreachable, even though the proof has full coverage of all truly reachable paths.
### Check for over-constrained inputs
Consider the case where one side of a branch is not reached, or where execution does not continue past an assumption.
In this case, it is possible that the inputs have been over-constrained
## How can I tell if my proof is over-constrained?
This will normally appear in coverage - overconstrained proofs will normally have unreachable portions of code.
You can also add a "smoke test", but adding assertions that you expect to fail to the code (which can be as simple as `assert(0)`).
If these assertions do not fail, then sometime is wrong with your proof.
## What should I do if CBMC crashes?
1. Make a new branch, containing the exact code that caused cbmc to crash.
We recommend giving it a name like `cbmc-crashing-bug-1`.
1. Push it to public github repo (if possible)
1. Post a bug report [here](https://github.com/diffblue/cbmc/issues/new), linking to the branch that you pushed containing the bug.
1. Post a bug report on this repo, linking to the bug that you posted on the main CBMC repo.

239
arcs-sdk/modules/coreHTTP/test/cbmc/aws-templates-for-cbmc-proofs/training-material/FAQ.md Normal file
View File

@@ -0,0 +1,239 @@
#CBMC FAQ
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
**Table of Contents**
- [How should I select the initial entry-point to verify?](#how-should-i-select-the-initial-entry-point-to-verify)
- [Top down approach](#top-down-approach)
- [Bottom up approach](#bottom-up-approach)
- [How do I set up the tools I need?](#how-do-i-set-up-the-tools-i-need)
- [How do I set up my repository for verification?](#how-do-i-set-up-my-repository-for-verification)
- [How do I write a good proof harness?](#how-do-i-write-a-good-proof-harness)
- [How do I write function pre/post conditions?](#how-do-i-write-function-prepost-conditions)
- [How do I write a proof Makefile?](#how-do-i-write-a-proof-makefile)
- [How do I write a good ensures function?](#how-do-i-write-a-good-ensures-function)
- [How should I write a good is_valid function?](#how-should-i-write-a-good-is_valid-function)
- [I see 12 proof failures. How do I select which one to debug?](#i-see-12-proof-failures-how-do-i-select-which-one-to-debug)
- [How do I debug a proof failure?](#how-do-i-debug-a-proof-failure)
- [What are some examples of good CBMC proofs?](#what-are-some-examples-of-good-cbmc-proofs)
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
## How should I select the initial entry-point to verify?
There are two basic approaches that we have used at AWS to verify C code using CBMC: **top down**, and **bottom up**.
In our experience, the top-down approach is best suited to a
### Top down approach
The top-down approach begins by selecting the most critical entry-points into the code.
For example, when verifying an HTTP library, you might choose to begin with the network parser that handles data directly off the wire.
Since this parser is directly exposed to untrusted input, verification leads to a significant security benefit.
* Advantages:
* Can be useful in cases where there is a limited amount of time to complete as much useful verification as possible, for example to before an upcoming feature release.
* Focus directly on the most security/safety critical portions of the code
* Disadvantages:
* Proofs can be very large, and tax the performance limits of CBMC
* Can require writing a large number of initializer functions
* Can require writing a large number of validity predicates
* Little opportunity to reuse work.
### Bottom up approach
The bottom-up approach follows the natural dependency flow of the codebase being verified.
In our experience, the more self-contained a piece of code is, the easier it is to verify.
1. Make a dependency graph of the modules in your program.
2. Select the leaves of the graph - those modules which other modules depend upon, but which do not depend on other modules themselves.
Typically, these include the basic data-structures and algorithms used by the rest of the codebase.
Which one of these you choose is a matter of style: you can use the [TODO, link to coding guidelines] coding guidelines to help select modules which are likely to be good verification targets.
3. Inside a given module, select the best initial verification target.
This is often, but not always, one of the simpler functions.
In particular, you are looking for a function which is both easy to verify, and will give good insight into the data-structure invariants of the data-structures used in the given module.
In our experience, it often makes sense to start with allocation or initialization functions (which often have named that end in `_alloc()` or `_init()`
## How do I set up the tools I need?
[TODO] - basically brew install / apt-get install. Link to that page.
## How do I set up my repository for verification?
We have created a "Proof Starter Kit" repository, which includes the basic information required to set up a build-system for your first proof.
1. Select where the proofs, and related artifacts, will go.
Most projects put them parallel to existing verification artifacts.[TODO, should we retire .cbmc_batch, and move it to tests/cbmc? for all projects]
For example, if your project has folders `project/tests/unit`, `projects/tests/integration`, and `projects/tests/fuzzing`, we would suggest adding a folder `project/tests/cbmc`.
1. Add the "CBMC Proof Starter Kit" repository, available at [TODO], to the `/cbmc` folder as a submodule.
Use the command `[TODO]`.
This will install a standard makefile, as well as a set of useful utility files.
Follow the documentation in that repository to set any needed project specific overrides in the makefile.
1. Create the folders:
1. `tests/cbmc/proofs`
1. `tests/cbmc/stubs`
1. `tests/cbmc/include`
1. `tests/cbmc/source`
1. Choose an initial proof target `functionname` following the instructions above.
We recommend choosing as simple a function as possible to verify in this case.
This forms a "tracer bullet" proof to ensure that your infrastructure is working correctly.
1. Create the folder `tests/cbmc/proofs/functionname`
1. Copy the sample proof makefile, and the sample proof harness, into that folder from the "proof starter kit" repo, and fill in the blanks.
How to actually write the proof is discussed below [TODO link]
1. Follow the instructions at [TODO where??] to enable CBMC CI for your repository
## How do I write a good proof harness?
We have developed a style of writing proofs that we believe is readable, maintainable, and modular.
This style was driven by feedback from developers, and addresses the need to communicate *exactly what we are proving* to developers and users.
Our proofs have the following features:
1. They are structured as *harnesses* that call into the function being verified, similar to unit tests.
This makes it easy to see how they work, as developers can `execute' the proof in their heads.
This style also yields more useful error traces.
1. They state their assumptions declaratively.
Rather than creating a fully-initialized data structure in imperative style, we create unconstrained data structures and then constrain them just enough to prove the property of interest.
This means the only assumptions on the data structure's values are the ones we state in the harness.
1. They follow a predictable pattern: setting up data structures, assuming preconditions on them, calling into the code being verified, and asserting postconditions.
The following code is an example of a proof harness:
```
void aws_array_list_get_at_ptr_harness() {
/* initialization */
struct aws_array_list list;
__CPROVER_assume(aws_array_list_is_bounded(&list));
ensure_array_list_has_allocated_data_member(&list);
/* generate unconstrained inputs */
void **val = can_fail_malloc(sizeof(void *));
size_t index;
/* preconditions */
__CPROVER_assume(aws_array_list_is_valid(&list));
__CPROVER_assume(val != NULL);
/* call function under verification */
if(!aws_array_list_get_at_ptr(&list, val, index)) {
/* If aws_array_list_get_at_ptr is successful,
* i.e. ret==0, we ensure the list isn't
* empty and index is within bounds */
assert(list.data != NULL);
assert(list.length > index);
}
/* postconditions */
assert(aws_array_list_is_valid(&list));
assert(val != NULL);
}
```
The harness shown above consists of five parts:
1. Initialize the data structure to unconstrained values.
We developed initializers for all verified data structures using a consistent naming scheme:
`ensure_{data_structure}_has_allocated_data_member()`.
1. Generate unconstrained inputs to the function.
1. Constrain all inputs to meet the function specification and assume all preconditions using `assume` statements.
If necessary, bound the data structures so that the proof terminates.
1. Call the function under verification with these inputs.
1. Check any function postconditions using `assert` statements.
This style of writing a proof harness is motivated by our desire to make assumptions explicit to developers.
This style consists of two steps.
The first step does the minimal work required to imperatively allocate
structures with unconstrained fields, as described in Items 1 and 2 in the above list.
The second step uses `assume` statements to enforce
the specification about the values that go in those fields (Item 3).
This makes the specification used in the proof harness clear and allows them to be further reused as assertions in the mainline code.
Syntactically, a proof harness looks quite similar to a unit test.
The main difference is that a proof harness calls the target function with a partially-constrained input rather than a concrete value; when symbolically executed by CBMC, this has the effect of exploring the function under *all* possible inputs that satisfy the constraints.
In fact, historically, we started from unit tests, and tried to make them symbolic by replacing concrete values with unconstrained values.
We found this difficult, since there are relations that constrain fields in a data structure and must be enforced (e.g., `length < capacity` and `capacity != 0 IMPLIES buffer != 0`).
Even worse, these imperative proof-harnesses turned out to be difficult to reason about and to explain to the development team.
## How do I write function pre/post conditions?
The preconditions used as assumptions in (Item $(3)$) are developed using an iterative process.
For each module, we start by specifying the simplest predicates that we can think of for the data structure --- usually, that the data of the data structure
is correctly allocated.
Then we gradually refine these predicates, until the development team accepts them as reasonable invariants for the data structure, aided by having all the unit and regression tests pass.
Using this process, we defined a set of predicates for each data structure in the C
source file so that they can be easily accessed and modified by the library
developers, and so that they serve as documentation for the library's users.
For instance, in the case of the `array_list`, we started
with the invariant that `data` points to `current_size`
allocated bytes.
After several iterations, the validity invariant for `array_list` ended up looking like this:
```
bool aws_array_list_is_valid(
const struct aws_array_list *list) {
if (!list) return false;
size_t required_size = 0;
bool required_size_is_valid =
(aws_mul_size_checked(list->length,
list->item_size,
&required_size)
== AWS_OP_SUCCESS);
bool current_size_is_valid =
(list->current_size >= required_size);
bool data_is_valid =
((list->current_size == 0 && list->data == NULL)
|| AWS_MEM_IS_WRITABLE(list->data, list->current_size));
bool item_size_is_valid = (list->item_size != 0);
return required_size_is_valid
&& current_size_is_valid
&& data_is_valid && item_size_is_valid;
}
```
The invariant above describes four conditions satisfied by a valid `array_list`:
1. the sum of the sizes of the items of the list must fit in an unsigned integer of type `size_t`, which is checked using the function `aws_mul_size_checked`
1. the size of the `array_list` in bytes (`current_size`) has to be larger than or equal to the sum of the sizes of its items;
1. the `data` pointer must point to a valid memory location, otherwise it must be `NULL` if the size of the `array_list` is zero;
1. the `item_size` must be positive.
## How do I write a proof Makefile?
## How do I write a good ensures function?
## How should I write a good is_valid function?
## I see 12 proof failures. How do I select which one to debug?
CBMC proof failures seem to come in batches: you run the proof, and see a dozen different errors reported
In many cases, these failures are related: instead of stressing about the number of failures, pick one, debug it, and see if fixing it removes (many of) the others.
Some good heuristics for deciding which failure to investigate:
1. Look for a failure that occurs early on in the proof.
This will often be the one with the shortest trace [TODO viewer should output this information].
The shorter the trace leading to the issue, the easier it is to debug.
1. Look for a failure in code you understand.
Some functions are simpler than others: a failure in a simple function is often easier to analyze that one in a complicated function.
And a failure in a function you understand is easier than one in
1. Look for a simple type of failure.
For example, the trace from a null dereference is often easier to follow than the trace for a use of a DEAD pointer.
But they're normally exactly the same bug!
## How do I debug a proof failure?
There are a number of techniques that have proven useful in debugging proof failures.
## What are some examples of good CBMC proofs?
[AWS-C-Common - Array List Copy](https://github.com/awslabs/aws-c-common/blob/master/.cbmc-batch/jobs/aws_array_list_copy/aws_array_list_copy_harness.c
)
[s2n: stuffer erase and read bytes](https://github.com/awslabs/s2n/blob/master/tests/cbmc/proofs/s2n_stuffer_erase_and_read_bytes/s2n_stuffer_erase_and_read_bytes_harness.c)

Some files were not shown because too many files have changed in this diff Show More