In your formula url field ->
"
https://www.google.com/maps/dir/" & [Starting Address] & "/" & [Ending Address]
If your starting address is static and will never change (like if you were always sending them from your office) you could swap out the field [Starting Address] and just type in your start like so:
"
https://www.google.com/maps/dir/1234 My Office Address, Denver, CO/" & [Ending Address]
If your starting address draws from another table like the campaign, or if you have an address unique to the Agent assigned like their home address they'd be traveling from, then pass it in as a lookup field and you'll be good to go like in the first example