Configuring Tools to Accept User-Provided Files

A key feature of interactive tools on allmates.ai is their ability to work with files that users attach to their messages. This guide explains how to configure a REST API Tool to correctly receive and process these files, whether the target API requires a URL or Base64-encoded data.

Understanding the Attachment Placeholder Mechanism

To simplify the process for the AI, the allmates.ai platform uses a powerful placeholder mechanism. Instead of forcing the Mate to handle raw file data, it only needs to use a specific placeholder string in the API request. At runtime, the platform automatically intercepts this placeholder and replaces it with the appropriate file data before sending the request to the final destination.

This approach offers significant benefits:

• Simplicity for the LLM: The AI doesn't need to read, encode, or manage file content. It only needs to construct a simple text placeholder.

• Security: Files are handled within the secure environment of the platform.

• Flexibility: The same mechanism supports both URL and Base64 methods, adaptable to any API's requirements.

The core of this system is the attachmentId, a unique identifier for each file uploaded by a user in a conversation, which is made available to the Mate in the context of the request.

Method 1: Passing Files via URL (Recommended)

This is the most efficient and common method. Use it when the external API is capable of downloading a file from a public URL.

How It Works

The platform generates a temporary, secure, and publicly accessible URL for the attached file. It then substitutes the placeholder with this URL.

Placeholder Syntax

{{mode:url|attachmentId:ATTACHMENT_ID}}

• mode:url: Instructs the platform to generate and use a file URL.

• attachmentId:ATTACHMENT_ID: The unique ID of the attached file.

OpenAPI Implementation Example

To implement this, you must provide clear instructions in the parameter's description for the LLM.

# In components/schemas/YourRequest/properties
image_url:
  type: string
  format: uri
  description: |
    URL of the source image to process.
    ## For LLMs
    - MUST contain the processing mode and the attachment ID.
    - The syntax should be of the form: {{mode:url|attachmentId:123456788999}} where url is the processing mode and 123456788999 is the attachment ID.
  example: '{{mode:url|attachmentId:123456788999}}'

Method 2: Passing Files as Base64 Data

Use this method when the external API requires the raw file data to be sent directly within the JSON request body.

How It Works

The platform reads the attached file, encodes its content into a Base64 string, and formats it as a complete Data URI (e.g., data:image/jpeg;base64,...). This entire Data URI replaces the placeholder.

Placeholder Syntax

{{mode:b64|attachmentId:ATTACHMENT_ID}}

• mode:b64: Instructs the platform to generate a Base64 Data URI.

• attachmentId:ATTACHMENT_ID: The unique ID of the attached file.

OpenAPI Implementation Example

The implementation is similar, but you instruct the AI to use the b64 mode.

# In components/schemas/YourRequest/properties
image_data:
  type: string
  format: uri  # A Data URI is a valid URI format
  description: |
    The source image, provided as a Base64-encoded Data URI.
    ## For LLMs
    - MUST contain the processing mode and the attachment ID.
    - The syntax should be of the form: {{mode:b64|attachmentId:123456788999}} where b64 is the processing mode and 123456788999 is the attachment ID.
  example: '{{mode:b64|attachmentId:123456788999}}'

Practical Example: Multi-File Upload

The placeholder mechanism is flexible enough to handle APIs that accept multiple files in a single call, such as in an array.

# In components/schemas/YourRequest/properties
images:
  type: array
  minItems: 1
  description: An array of source images to combine.
  items:
    type: object
    required: [url]
    properties:
      url:
        type: string
        format: uri
        description: |
          A source image to be used in the operation.
          ## For LLMs
          - Use one placeholder for each attached image, providing its unique attachmentId.
          - The syntax is: {{mode:url|attachmentId:ATTACHMENT_ID}}
        example: '{{mode:url|attachmentId:9876543210}}'

Best Practices for Handling File Uploads

1. Prefer the URL Method: Unless the target API documentation explicitly requires Base64 data, always prefer using mode:url. It is more performant as it keeps the JSON payload small.

2. Be Explicit in LLM Instructions: The ## For LLMs section is critical. Always include the exact placeholder syntax ({{mode:url|attachmentId:ID}} or {{mode:b64|attachmentId:ID}}) and state clearly that ATTACHMENT_ID must be replaced with the ID provided in the conversation context.

3. Validate the Target API's Requirements: Before designing your schema, thoroughly check the external API's documentation. Does it expect a URL or Base64 data? Does it expect a Data URI prefix? This will determine which mode to use.

4. Use Descriptive operationIds: Ensure your operation IDs (e.g., uploadDocument, editImageWithText) clearly communicate the action, helping the AI choose the correct tool and operation.

Conclusion

Mastering the attachment placeholder mechanism is essential for creating powerful, interactive tools that can work with user-provided files. By correctly configuring your OpenAPI schema with clear instructions for the LLM, you can seamlessly integrate file upload capabilities, allowing your Mates to edit images, analyze documents, and perform a wide range of tasks on user-provided content.