We need to keep in mind, however, that tillage breaks soil structure, and does never build it. So you also have to work on building soil structure and avoiding causing compaction again after you do any tillage.

A compacted layer can be diagnosed with a penetrometer. You will notice the penetration resistance becoming very high (>300 psi) and then decreasing after you push the penetrometer through it.

Determine how deep the bottom of the compacted layer is and set your subsoiler or chisel plow one inch deeper than that.

Make sure the soil is in optimal condition to shatter it. If it is too wet you are likely do more harm than good by smearing up the soil next to the shanks.

The choice of shanks is also important. You want shanks that do as little surface disturbance as possible to maintain optimal residue cover.

Curved or parabolic shanks cause more surface disturbance than straight shanks, but are easier to pull.

Good results have been achieved with paraplow shanks that curve sideways. They do not disturb the surface much but cause maximum shattering below ground, and take less power to pull than straight leg subsoilers.

Shanks should be narrow to avoid disturbing the soil surface much.

Shank tips are also important. The larger the tip, the more shattering, but one has to watch out with bringing too much soil to the surface.

Kickback-mechanisms are essential if you have rocks of tree roots in the soil. If not, you will be busting up shanks or breaking lots of shear bolts! 

Make sure your tractor can pull the subsoiler. In general, you need a minimum of 40 tractor horsepower per subsoiler shank.

Again, once you have completed a subsoiling operation, make sure you rebuild soil structure by planting a (cover) crop with heavy root mass, and add organic matter. After this remedial action, you want to design your management to avoid causing compaction in the future.

More info on avoiding soil compaction can be found in this Penn State fact sheet: http://www.extension.psu.edu/pubs/uc186.