Ben's is a local landmark more notable for its history (celebrity clientele, survived the 1968 riots while everything around it was destroyed) and late-night diner experience than it is for the quality of the food, which is average. If Ben's were to open for the first time today, you'd find it to be your typical greasy-spoon late-night diner. I don't think anybody in the world would say "this is the best chili I've ever had," but there's something that numbs our tastebuds when the environment is right. Ben's is one of those places.
For most visitors to Washington, I think you'll find it's off the beaten path and frankly not worth the effort to get there for the food alone.