We discuss local elimination algorithms that compute global information using local computations. Results of benchmarking show real computational capabilities of block elimination algorithms combined with SYMPHONY solver. Strategies for parallelizing a sequential local elimination algorithm for sparse discrete optimization problems are analyzed. We propose to use hybrid Master-Worker scheme where Worker processors (GPUs) solve concurrently subproblems corresponding to super-nodes of extended elimination tree that are generated by a single master process (CPU).