To customize sidebar or footer widgets, navigate to Appearance -> Widgets. Here you can see all the sidebars: Blog Sidebar, Page Sidebar and Shop Sidebar in case if WooCommerce is installed. Drag the widgets to your sidebars and arrange them if needed.
