Soak herring in cold water for at least 1 hour. If very salty, repeat, changing the water each time. Drain thoroughly and slice herring into bite-size pieces. Place in a jar large enough to accommodate the pieces and cover with oil, allspice, peppercorns, and bay leaf. Close the jar. Refrigerate for 2 to 3 days before eating. This will keep refrigerated up to 2 weeks. Serve with finely chopped onion or onion slices, lemon, and parsley or dill.