python - pubsub publisher retry settings failing with: TypeError: All attributes being published to Pub/Sub must be sent as text strings

Question

I have a cloud function which is errors when using the pubsub publisher retry settings on the publisher client or publish requests. https://cloud.google.com/pubsub/docs/samples/pubsub-publisher-retry-settings#pubsub_publisher_retry_settings-python

When i run my code in JupterLab the python code runs successfully but as soon as I move the code to Cloud Functions I get a TypeError: All attributes being published to Pub/Sub must be sent as text strings.

I have now tried a new simple Cloud Function copying the code directly from the example in the like above link but still get the same error, any suggestions much appreaciated.

from google.cloud import pubsub_v1

# TODO(developer)
GCP_PROJECT_ID='test_project'
SIT_EVENT_TOPIC = ('test_project1')
topic_id=SIT_EVENT_TOPIC
project_id=GCP_PROJECT_ID
# project_id = "your-project-id"
# topic_id = "your-topic-id"

# Configure the retry settings. Defaults shown in comments are values applied
# by the library by default, instead of default values in the Retry object.
def test():
    custom_retry = api_core.retry.Retry(
        initial=0.250,  # seconds (default: 0.1)
        maximum=90.0,  # seconds (default: 60.0)
        multiplier=1.45,  # default: 1.3
        deadline=300.0,  # seconds (default: 60.0)
        predicate=api_core.retry.if_exception_type(
            api_core.exceptions.Aborted,
            api_core.exceptions.DeadlineExceeded,
            api_core.exceptions.InternalServerError,
            api_core.exceptions.ResourceExhausted,
            api_core.exceptions.ServiceUnavailable,
            api_core.exceptions.Unknown,
            api_core.exceptions.Cancelled,
        ),
    )

    publisher = pubsub_v1.PublisherClient()
    topic_path = publisher.topic_path(project_id, topic_id)

    # for n in range(1, 10):
    #     data = "Message number {}".format(n)
    data="test message"
    # Data must be a bytestring
    data = data.encode("utf-8")
    print(type(data))
    future = publisher.publish(topic=topic_path, data=data, retry=custom_retry)
    print(future.result())

    print(f"Published messages with retry settings to {topic_path}.")


def main(event, context):
    """
    Call the main function, sets the order in which to run functions.
    """
    test()

    return 'Script has run without errors !!'

if (__name__ == "__main__"):
    main()

output

ine 40, in test future = publisher.publish(topic=topic_path, data=data, retry=custom_retry) File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/cloud/pubsub_v1/publisher/client.py", line 195, in publish raise TypeError( TypeError: All attributes being published to Pub/Sub must be sent as text strings.

Answer 1

This issue was caused by running outdated python packages in the requirements.txt file. ?

Original requirements list

google-cloud-pubsub==0.34.0

google-cloud-storage==1.13.1

google-cloud-bigquery==1.8.1

google-cloud-core==0.29.1

ndjson==0.3.1

?

New requirements list

google-cloud-pubsub==2.2.0

google-cloud-core==1.5.0

google-api-core==1.24.1

google-cloud-storage==1.35.0

google-cloud-bigquery==2.6.2

ndjson==0.3.1

Updating the list allowed the cloud function to run, another issue I found with the example code is that you need to convert the data to a string, I used the following prior to publishing:

data=str(data)