While centralized approaches to multi-vehicle routing problems typically provide provably optimal solutions, they do not scale well. In this paper, an algorithm for decentralized multi-vehicle routing is introduced that is often associated with significantly lower computational demands, but does not sacrifice the optimality of the found solution. In particular, we consider a fleet of autonomous vehicles traversing a road network that need to service a potentially infinite set of gradually appearing travel requests specified by their pick-up and drop-off points. The proposed algorithm synthesizes optimal assignment of the travel requests to the vehicles as well as optimal routes by utilizing Fast Marching Method (FMM) that restricts the search for the optimal assignment to a local subnetwork as opposed to the global road network. Several illustrative case studies are presented to demonstrate the effectiveness and efficiency of the approach.