In scalable video coding, multiple versions of the same video which can differ in resolution, bitrate and framerate are encoded into one scalable bitstream which allows for an efficient compression of all representations. While the latest scalable video coding standard Scalable High Efficiency Video Coding (SHVC) enables a highly efficient compression it also significantly increases the decoder complexity compared to non-scalable coding in case of quality scalability. In this work, SHVC is extended by a flexible inter layer prediction scheme and an inter layer refinement method which allows for a binary mapping between arbitrary quantizer step sizes between the layers. It is shown how all of these modifications can significantly reduce the required decoder complexity to much more reasonable values.