summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
-rw-r--r--Documentation/livepatch/livepatch.txt135
-rw-r--r--include/linux/livepatch.h7
-rw-r--r--kernel/livepatch/core.c280
-rw-r--r--kernel/livepatch/core.h2
-rw-r--r--kernel/livepatch/transition.c19
-rw-r--r--samples/livepatch/livepatch-callbacks-demo.c13
-rw-r--r--samples/livepatch/livepatch-sample.c13
-rw-r--r--samples/livepatch/livepatch-shadow-fix1.c14
-rw-r--r--samples/livepatch/livepatch-shadow-fix2.c14
9 files changed, 168 insertions, 329 deletions
diff --git a/Documentation/livepatch/livepatch.txt b/Documentation/livepatch/livepatch.txt
index 2d7ed09dbd59..8f56490a4bb6 100644
--- a/Documentation/livepatch/livepatch.txt
+++ b/Documentation/livepatch/livepatch.txt
@@ -12,12 +12,11 @@ Table of Contents:
124. Livepatch module 124. Livepatch module
13 4.1. New functions 13 4.1. New functions
14 4.2. Metadata 14 4.2. Metadata
15 4.3. Livepatch module handling
165. Livepatch life-cycle 155. Livepatch life-cycle
17 5.1. Registration 16 5.1. Loading
18 5.2. Enabling 17 5.2. Enabling
19 5.3. Disabling 18 5.3. Disabling
20 5.4. Unregistration 19 5.4. Removing
216. Sysfs 206. Sysfs
227. Limitations 217. Limitations
23 22
@@ -298,117 +297,89 @@ into three levels:
298 see the "Consistency model" section. 297 see the "Consistency model" section.
299 298
300 299
3014.3. Livepatch module handling
302------------------------------
303
304The usual behavior is that the new functions will get used when
305the livepatch module is loaded. For this, the module init() function
306has to register the patch (struct klp_patch) and enable it. See the
307section "Livepatch life-cycle" below for more details about these
308two operations.
309
310Module removal is only safe when there are no users of the underlying
311functions. This is the reason why the force feature permanently disables
312the removal. The forced tasks entered the functions but we cannot say
313that they returned back. Therefore it cannot be decided when the
314livepatch module can be safely removed. When the system is successfully
315transitioned to a new patch state (patched/unpatched) without being
316forced it is guaranteed that no task sleeps or runs in the old code.
317
318
3195. Livepatch life-cycle 3005. Livepatch life-cycle
320======================= 301=======================
321 302
322Livepatching defines four basic operations that define the life cycle of each 303Livepatching can be described by four basic operations:
323live patch: registration, enabling, disabling and unregistration. There are 304loading, enabling, disabling, removing.
324several reasons why it is done this way.
325
326First, the patch is applied only when all patched symbols for already
327loaded objects are found. The error handling is much easier if this
328check is done before particular functions get redirected.
329 305
330Second, it might take some time until the entire system is migrated with
331the hybrid consistency model being used. The patch revert might block
332the livepatch module removal for too long. Therefore it is useful to
333revert the patch using a separate operation that might be called
334explicitly. But it does not make sense to remove all information until
335the livepatch module is really removed.
336 306
3075.1. Loading
308------------
337 309
3385.1. Registration 310The only reasonable way is to enable the patch when the livepatch kernel
339----------------- 311module is being loaded. For this, klp_enable_patch() has to be called
312in the module_init() callback. There are two main reasons:
340 313
341Each patch first has to be registered using klp_register_patch(). This makes 314First, only the module has an easy access to the related struct klp_patch.
342the patch known to the livepatch framework. Also it does some preliminary
343computing and checks.
344 315
345In particular, the patch is added into the list of known patches. The 316Second, the error code might be used to refuse loading the module when
346addresses of the patched functions are found according to their names. 317the patch cannot get enabled.
347The special relocations, mentioned in the section "New functions", are
348applied. The relevant entries are created under
349/sys/kernel/livepatch/<name>. The patch is rejected when any operation
350fails.
351 318
352 319
3535.2. Enabling 3205.2. Enabling
354------------- 321-------------
355 322
356Registered patches might be enabled either by calling klp_enable_patch() or 323The livepatch gets enabled by calling klp_enable_patch() from
357by writing '1' to /sys/kernel/livepatch/<name>/enabled. The system will 324the module_init() callback. The system will start using the new
358start using the new implementation of the patched functions at this stage. 325implementation of the patched functions at this stage.
359 326
360When a patch is enabled, livepatch enters into a transition state where 327First, the addresses of the patched functions are found according to their
361tasks are converging to the patched state. This is indicated by a value 328names. The special relocations, mentioned in the section "New functions",
362of '1' in /sys/kernel/livepatch/<name>/transition. Once all tasks have 329are applied. The relevant entries are created under
363been patched, the 'transition' value changes to '0'. For more 330/sys/kernel/livepatch/<name>. The patch is rejected when any above
364information about this process, see the "Consistency model" section. 331operation fails.
365 332
366If an original function is patched for the first time, a function 333Second, livepatch enters into a transition state where tasks are converging
367specific struct klp_ops is created and an universal ftrace handler is 334to the patched state. If an original function is patched for the first
368registered. 335time, a function specific struct klp_ops is created and an universal
336ftrace handler is registered[*]. This stage is indicated by a value of '1'
337in /sys/kernel/livepatch/<name>/transition. For more information about
338this process, see the "Consistency model" section.
369 339
370Functions might be patched multiple times. The ftrace handler is registered 340Finally, once all tasks have been patched, the 'transition' value changes
371only once for the given function. Further patches just add an entry to the 341to '0'.
372list (see field `func_stack`) of the struct klp_ops. The last added
373entry is chosen by the ftrace handler and becomes the active function
374replacement.
375 342
376Note that the patches might be enabled in a different order than they were 343[*] Note that functions might be patched multiple times. The ftrace handler
377registered. 344 is registered only once for a given function. Further patches just add
345 an entry to the list (see field `func_stack`) of the struct klp_ops.
346 The right implementation is selected by the ftrace handler, see
347 the "Consistency model" section.
378 348
379 349
3805.3. Disabling 3505.3. Disabling
381-------------- 351--------------
382 352
383Enabled patches might get disabled either by calling klp_disable_patch() or 353Enabled patches might get disabled by writing '0' to
384by writing '0' to /sys/kernel/livepatch/<name>/enabled. At this stage 354/sys/kernel/livepatch/<name>/enabled.
385either the code from the previously enabled patch or even the original
386code gets used.
387 355
388When a patch is disabled, livepatch enters into a transition state where 356First, livepatch enters into a transition state where tasks are converging
389tasks are converging to the unpatched state. This is indicated by a 357to the unpatched state. The system starts using either the code from
390value of '1' in /sys/kernel/livepatch/<name>/transition. Once all tasks 358the previously enabled patch or even the original one. This stage is
391have been unpatched, the 'transition' value changes to '0'. For more 359indicated by a value of '1' in /sys/kernel/livepatch/<name>/transition.
392information about this process, see the "Consistency model" section. 360For more information about this process, see the "Consistency model"
361section.
393 362
394Here all the functions (struct klp_func) associated with the to-be-disabled 363Second, once all tasks have been unpatched, the 'transition' value changes
364to '0'. All the functions (struct klp_func) associated with the to-be-disabled
395patch are removed from the corresponding struct klp_ops. The ftrace handler 365patch are removed from the corresponding struct klp_ops. The ftrace handler
396is unregistered and the struct klp_ops is freed when the func_stack list 366is unregistered and the struct klp_ops is freed when the func_stack list
397becomes empty. 367becomes empty.
398 368
399Patches must be disabled in exactly the reverse order in which they were 369Third, the sysfs interface is destroyed.
400enabled. It makes the problem and the implementation much easier.
401 370
371Note that patches must be disabled in exactly the reverse order in which
372they were enabled. It makes the problem and the implementation much easier.
402 373
4035.4. Unregistration
404-------------------
405 374
406Disabled patches might be unregistered by calling klp_unregister_patch(). 3755.4. Removing
407This can be done only when the patch is disabled and the code is no longer 376-------------
408used. It must be called before the livepatch module gets unloaded.
409 377
410At this stage, all the relevant sys-fs entries are removed and the patch 378Module removal is only safe when there are no users of functions provided
411is removed from the list of known patches. 379by the module. This is the reason why the force feature permanently
380disables the removal. Only when the system is successfully transitioned
381to a new patch state (patched/unpatched) without being forced it is
382guaranteed that no task sleeps or runs in the old code.
412 383
413 384
4146. Sysfs 3856. Sysfs
diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
index 6a9165d9b090..8f9c19c69744 100644
--- a/include/linux/livepatch.h
+++ b/include/linux/livepatch.h
@@ -139,11 +139,12 @@ struct klp_object {
139 * struct klp_patch - patch structure for live patching 139 * struct klp_patch - patch structure for live patching
140 * @mod: reference to the live patch module 140 * @mod: reference to the live patch module
141 * @objs: object entries for kernel objects to be patched 141 * @objs: object entries for kernel objects to be patched
142 * @list: list node for global list of registered patches 142 * @list: list node for global list of actively used patches
143 * @kobj: kobject for sysfs resources 143 * @kobj: kobject for sysfs resources
144 * @kobj_added: @kobj has been added and needs freeing 144 * @kobj_added: @kobj has been added and needs freeing
145 * @enabled: the patch is enabled (but operation may be incomplete) 145 * @enabled: the patch is enabled (but operation may be incomplete)
146 * @forced: was involved in a forced transition 146 * @forced: was involved in a forced transition
147 * @free_work: patch cleanup from workqueue-context
147 * @finish: for waiting till it is safe to remove the patch module 148 * @finish: for waiting till it is safe to remove the patch module
148 */ 149 */
149struct klp_patch { 150struct klp_patch {
@@ -157,6 +158,7 @@ struct klp_patch {
157 bool kobj_added; 158 bool kobj_added;
158 bool enabled; 159 bool enabled;
159 bool forced; 160 bool forced;
161 struct work_struct free_work;
160 struct completion finish; 162 struct completion finish;
161}; 163};
162 164
@@ -168,10 +170,7 @@ struct klp_patch {
168 func->old_name || func->new_func || func->old_sympos; \ 170 func->old_name || func->new_func || func->old_sympos; \
169 func++) 171 func++)
170 172
171int klp_register_patch(struct klp_patch *);
172int klp_unregister_patch(struct klp_patch *);
173int klp_enable_patch(struct klp_patch *); 173int klp_enable_patch(struct klp_patch *);
174int klp_disable_patch(struct klp_patch *);
175 174
176void arch_klp_init_object_loaded(struct klp_patch *patch, 175void arch_klp_init_object_loaded(struct klp_patch *patch,
177 struct klp_object *obj); 176 struct klp_object *obj);
diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
index e77c5017ae0c..bd41b03a72d5 100644
--- a/kernel/livepatch/core.c
+++ b/kernel/livepatch/core.c
@@ -45,7 +45,11 @@
45 */ 45 */
46DEFINE_MUTEX(klp_mutex); 46DEFINE_MUTEX(klp_mutex);
47 47
48/* Registered patches */ 48/*
49 * Actively used patches: enabled or in transition. Note that replaced
50 * or disabled patches are not listed even though the related kernel
51 * module still can be loaded.
52 */
49LIST_HEAD(klp_patches); 53LIST_HEAD(klp_patches);
50 54
51static struct kobject *klp_root_kobj; 55static struct kobject *klp_root_kobj;
@@ -83,17 +87,6 @@ static void klp_find_object_module(struct klp_object *obj)
83 mutex_unlock(&module_mutex); 87 mutex_unlock(&module_mutex);
84} 88}
85 89
86static bool klp_is_patch_registered(struct klp_patch *patch)
87{
88 struct klp_patch *mypatch;
89
90 list_for_each_entry(mypatch, &klp_patches, list)
91 if (mypatch == patch)
92 return true;
93
94 return false;
95}
96
97static bool klp_initialized(void) 90static bool klp_initialized(void)
98{ 91{
99 return !!klp_root_kobj; 92 return !!klp_root_kobj;
@@ -292,7 +285,6 @@ static int klp_write_object_relocations(struct module *pmod,
292 * /sys/kernel/livepatch/<patch>/<object>/<function,sympos> 285 * /sys/kernel/livepatch/<patch>/<object>/<function,sympos>
293 */ 286 */
294static int __klp_disable_patch(struct klp_patch *patch); 287static int __klp_disable_patch(struct klp_patch *patch);
295static int __klp_enable_patch(struct klp_patch *patch);
296 288
297static ssize_t enabled_store(struct kobject *kobj, struct kobj_attribute *attr, 289static ssize_t enabled_store(struct kobject *kobj, struct kobj_attribute *attr,
298 const char *buf, size_t count) 290 const char *buf, size_t count)
@@ -309,40 +301,32 @@ static ssize_t enabled_store(struct kobject *kobj, struct kobj_attribute *attr,
309 301
310 mutex_lock(&klp_mutex); 302 mutex_lock(&klp_mutex);
311 303
312 if (!klp_is_patch_registered(patch)) {
313 /*
314 * Module with the patch could either disappear meanwhile or is
315 * not properly initialized yet.
316 */
317 ret = -EINVAL;
318 goto err;
319 }
320
321 if (patch->enabled == enabled) { 304 if (patch->enabled == enabled) {
322 /* already in requested state */ 305 /* already in requested state */
323 ret = -EINVAL; 306 ret = -EINVAL;
324 goto err; 307 goto out;
325 } 308 }
326 309
327 if (patch == klp_transition_patch) { 310 /*
311 * Allow to reverse a pending transition in both ways. It might be
312 * necessary to complete the transition without forcing and breaking
313 * the system integrity.
314 *
315 * Do not allow to re-enable a disabled patch.
316 */
317 if (patch == klp_transition_patch)
328 klp_reverse_transition(); 318 klp_reverse_transition();
329 } else if (enabled) { 319 else if (!enabled)
330 ret = __klp_enable_patch(patch);
331 if (ret)
332 goto err;
333 } else {
334 ret = __klp_disable_patch(patch); 320 ret = __klp_disable_patch(patch);
335 if (ret) 321 else
336 goto err; 322 ret = -EINVAL;
337 }
338 323
324out:
339 mutex_unlock(&klp_mutex); 325 mutex_unlock(&klp_mutex);
340 326
327 if (ret)
328 return ret;
341 return count; 329 return count;
342
343err:
344 mutex_unlock(&klp_mutex);
345 return ret;
346} 330}
347 331
348static ssize_t enabled_show(struct kobject *kobj, 332static ssize_t enabled_show(struct kobject *kobj,
@@ -508,7 +492,7 @@ static void klp_free_objects(struct klp_patch *patch)
508 * The operation must be completed by calling klp_free_patch_finish() 492 * The operation must be completed by calling klp_free_patch_finish()
509 * outside klp_mutex. 493 * outside klp_mutex.
510 */ 494 */
511static void klp_free_patch_start(struct klp_patch *patch) 495void klp_free_patch_start(struct klp_patch *patch)
512{ 496{
513 if (!list_empty(&patch->list)) 497 if (!list_empty(&patch->list))
514 list_del(&patch->list); 498 list_del(&patch->list);
@@ -536,6 +520,23 @@ static void klp_free_patch_finish(struct klp_patch *patch)
536 kobject_put(&patch->kobj); 520 kobject_put(&patch->kobj);
537 wait_for_completion(&patch->finish); 521 wait_for_completion(&patch->finish);
538 } 522 }
523
524 /* Put the module after the last access to struct klp_patch. */
525 if (!patch->forced)
526 module_put(patch->mod);
527}
528
529/*
530 * The livepatch might be freed from sysfs interface created by the patch.
531 * This work allows to wait until the interface is destroyed in a separate
532 * context.
533 */
534static void klp_free_patch_work_fn(struct work_struct *work)
535{
536 struct klp_patch *patch =
537 container_of(work, struct klp_patch, free_work);
538
539 klp_free_patch_finish(patch);
539} 540}
540 541
541static int klp_init_func(struct klp_object *obj, struct klp_func *func) 542static int klp_init_func(struct klp_object *obj, struct klp_func *func)
@@ -661,6 +662,7 @@ static int klp_init_patch_early(struct klp_patch *patch)
661 patch->kobj_added = false; 662 patch->kobj_added = false;
662 patch->enabled = false; 663 patch->enabled = false;
663 patch->forced = false; 664 patch->forced = false;
665 INIT_WORK(&patch->free_work, klp_free_patch_work_fn);
664 init_completion(&patch->finish); 666 init_completion(&patch->finish);
665 667
666 klp_for_each_object(patch, obj) { 668 klp_for_each_object(patch, obj) {
@@ -673,6 +675,9 @@ static int klp_init_patch_early(struct klp_patch *patch)
673 func->kobj_added = false; 675 func->kobj_added = false;
674 } 676 }
675 677
678 if (!try_module_get(patch->mod))
679 return -ENODEV;
680
676 return 0; 681 return 0;
677} 682}
678 683
@@ -681,115 +686,22 @@ static int klp_init_patch(struct klp_patch *patch)
681 struct klp_object *obj; 686 struct klp_object *obj;
682 int ret; 687 int ret;
683 688
684 mutex_lock(&klp_mutex);
685
686 ret = klp_init_patch_early(patch);
687 if (ret) {
688 mutex_unlock(&klp_mutex);
689 return ret;
690 }
691
692 ret = kobject_init_and_add(&patch->kobj, &klp_ktype_patch, 689 ret = kobject_init_and_add(&patch->kobj, &klp_ktype_patch,
693 klp_root_kobj, "%s", patch->mod->name); 690 klp_root_kobj, "%s", patch->mod->name);
694 if (ret) { 691 if (ret)
695 mutex_unlock(&klp_mutex);
696 return ret; 692 return ret;
697 }
698 patch->kobj_added = true; 693 patch->kobj_added = true;
699 694
700 klp_for_each_object(patch, obj) { 695 klp_for_each_object(patch, obj) {
701 ret = klp_init_object(patch, obj); 696 ret = klp_init_object(patch, obj);
702 if (ret) 697 if (ret)
703 goto free; 698 return ret;
704 } 699 }
705 700
706 list_add_tail(&patch->list, &klp_patches); 701 list_add_tail(&patch->list, &klp_patches);
707 702
708 mutex_unlock(&klp_mutex);
709
710 return 0;
711
712free:
713 klp_free_patch_start(patch);
714
715 mutex_unlock(&klp_mutex);
716
717 klp_free_patch_finish(patch);
718
719 return ret;
720}
721
722/**
723 * klp_unregister_patch() - unregisters a patch
724 * @patch: Disabled patch to be unregistered
725 *
726 * Frees the data structures and removes the sysfs interface.
727 *
728 * Return: 0 on success, otherwise error
729 */
730int klp_unregister_patch(struct klp_patch *patch)
731{
732 int ret;
733
734 mutex_lock(&klp_mutex);
735
736 if (!klp_is_patch_registered(patch)) {
737 ret = -EINVAL;
738 goto err;
739 }
740
741 if (patch->enabled) {
742 ret = -EBUSY;
743 goto err;
744 }
745
746 klp_free_patch_start(patch);
747
748 mutex_unlock(&klp_mutex);
749
750 klp_free_patch_finish(patch);
751
752 return 0; 703 return 0;
753err:
754 mutex_unlock(&klp_mutex);
755 return ret;
756}
757EXPORT_SYMBOL_GPL(klp_unregister_patch);
758
759/**
760 * klp_register_patch() - registers a patch
761 * @patch: Patch to be registered
762 *
763 * Initializes the data structure associated with the patch and
764 * creates the sysfs interface.
765 *
766 * There is no need to take the reference on the patch module here. It is done
767 * later when the patch is enabled.
768 *
769 * Return: 0 on success, otherwise error
770 */
771int klp_register_patch(struct klp_patch *patch)
772{
773 if (!patch || !patch->mod)
774 return -EINVAL;
775
776 if (!is_livepatch_module(patch->mod)) {
777 pr_err("module %s is not marked as a livepatch module\n",
778 patch->mod->name);
779 return -EINVAL;
780 }
781
782 if (!klp_initialized())
783 return -ENODEV;
784
785 if (!klp_have_reliable_stack()) {
786 pr_err("This architecture doesn't have support for the livepatch consistency model.\n");
787 return -ENOSYS;
788 }
789
790 return klp_init_patch(patch);
791} 704}
792EXPORT_SYMBOL_GPL(klp_register_patch);
793 705
794static int __klp_disable_patch(struct klp_patch *patch) 706static int __klp_disable_patch(struct klp_patch *patch)
795{ 707{
@@ -802,8 +714,7 @@ static int __klp_disable_patch(struct klp_patch *patch)
802 return -EBUSY; 714 return -EBUSY;
803 715
804 /* enforce stacking: only the last enabled patch can be disabled */ 716 /* enforce stacking: only the last enabled patch can be disabled */
805 if (!list_is_last(&patch->list, &klp_patches) && 717 if (!list_is_last(&patch->list, &klp_patches))
806 list_next_entry(patch, list)->enabled)
807 return -EBUSY; 718 return -EBUSY;
808 719
809 klp_init_transition(patch, KLP_UNPATCHED); 720 klp_init_transition(patch, KLP_UNPATCHED);
@@ -822,44 +733,12 @@ static int __klp_disable_patch(struct klp_patch *patch)
822 smp_wmb(); 733 smp_wmb();
823 734
824 klp_start_transition(); 735 klp_start_transition();
825 klp_try_complete_transition();
826 patch->enabled = false; 736 patch->enabled = false;
737 klp_try_complete_transition();
827 738
828 return 0; 739 return 0;
829} 740}
830 741
831/**
832 * klp_disable_patch() - disables a registered patch
833 * @patch: The registered, enabled patch to be disabled
834 *
835 * Unregisters the patched functions from ftrace.
836 *
837 * Return: 0 on success, otherwise error
838 */
839int klp_disable_patch(struct klp_patch *patch)
840{
841 int ret;
842
843 mutex_lock(&klp_mutex);
844
845 if (!klp_is_patch_registered(patch)) {
846 ret = -EINVAL;
847 goto err;
848 }
849
850 if (!patch->enabled) {
851 ret = -EINVAL;
852 goto err;
853 }
854
855 ret = __klp_disable_patch(patch);
856
857err:
858 mutex_unlock(&klp_mutex);
859 return ret;
860}
861EXPORT_SYMBOL_GPL(klp_disable_patch);
862
863static int __klp_enable_patch(struct klp_patch *patch) 742static int __klp_enable_patch(struct klp_patch *patch)
864{ 743{
865 struct klp_object *obj; 744 struct klp_object *obj;
@@ -871,17 +750,8 @@ static int __klp_enable_patch(struct klp_patch *patch)
871 if (WARN_ON(patch->enabled)) 750 if (WARN_ON(patch->enabled))
872 return -EINVAL; 751 return -EINVAL;
873 752
874 /* enforce stacking: only the first disabled patch can be enabled */ 753 if (!patch->kobj_added)
875 if (patch->list.prev != &klp_patches && 754 return -EINVAL;
876 !list_prev_entry(patch, list)->enabled)
877 return -EBUSY;
878
879 /*
880 * A reference is taken on the patch module to prevent it from being
881 * unloaded.
882 */
883 if (!try_module_get(patch->mod))
884 return -ENODEV;
885 755
886 pr_notice("enabling patch '%s'\n", patch->mod->name); 756 pr_notice("enabling patch '%s'\n", patch->mod->name);
887 757
@@ -916,8 +786,8 @@ static int __klp_enable_patch(struct klp_patch *patch)
916 } 786 }
917 787
918 klp_start_transition(); 788 klp_start_transition();
919 klp_try_complete_transition();
920 patch->enabled = true; 789 patch->enabled = true;
790 klp_try_complete_transition();
921 791
922 return 0; 792 return 0;
923err: 793err:
@@ -928,11 +798,15 @@ err:
928} 798}
929 799
930/** 800/**
931 * klp_enable_patch() - enables a registered patch 801 * klp_enable_patch() - enable the livepatch
932 * @patch: The registered, disabled patch to be enabled 802 * @patch: patch to be enabled
933 * 803 *
934 * Performs the needed symbol lookups and code relocations, 804 * Initializes the data structure associated with the patch, creates the sysfs
935 * then registers the patched functions with ftrace. 805 * interface, performs the needed symbol lookups and code relocations,
806 * registers the patched functions with ftrace.
807 *
808 * This function is supposed to be called from the livepatch module_init()
809 * callback.
936 * 810 *
937 * Return: 0 on success, otherwise error 811 * Return: 0 on success, otherwise error
938 */ 812 */
@@ -940,17 +814,51 @@ int klp_enable_patch(struct klp_patch *patch)
940{ 814{
941 int ret; 815 int ret;
942 816
817 if (!patch || !patch->mod)
818 return -EINVAL;
819
820 if (!is_livepatch_module(patch->mod)) {
821 pr_err("module %s is not marked as a livepatch module\n",
822 patch->mod->name);
823 return -EINVAL;
824 }
825
826 if (!klp_initialized())
827 return -ENODEV;
828
829 if (!klp_have_reliable_stack()) {
830 pr_err("This architecture doesn't have support for the livepatch consistency model.\n");
831 return -ENOSYS;
832 }
833
834
943 mutex_lock(&klp_mutex); 835 mutex_lock(&klp_mutex);
944 836
945 if (!klp_is_patch_registered(patch)) { 837 ret = klp_init_patch_early(patch);
946 ret = -EINVAL; 838 if (ret) {
947 goto err; 839 mutex_unlock(&klp_mutex);
840 return ret;
948 } 841 }
949 842
843 ret = klp_init_patch(patch);
844 if (ret)
845 goto err;
846
950 ret = __klp_enable_patch(patch); 847 ret = __klp_enable_patch(patch);
848 if (ret)
849 goto err;
850
851 mutex_unlock(&klp_mutex);
852
853 return 0;
951 854
952err: 855err:
856 klp_free_patch_start(patch);
857
953 mutex_unlock(&klp_mutex); 858 mutex_unlock(&klp_mutex);
859
860 klp_free_patch_finish(patch);
861
954 return ret; 862 return ret;
955} 863}
956EXPORT_SYMBOL_GPL(klp_enable_patch); 864EXPORT_SYMBOL_GPL(klp_enable_patch);
diff --git a/kernel/livepatch/core.h b/kernel/livepatch/core.h
index d0cb5390e247..d4eefc520c08 100644
--- a/kernel/livepatch/core.h
+++ b/kernel/livepatch/core.h
@@ -7,6 +7,8 @@
7extern struct mutex klp_mutex; 7extern struct mutex klp_mutex;
8extern struct list_head klp_patches; 8extern struct list_head klp_patches;
9 9
10void klp_free_patch_start(struct klp_patch *patch);
11
10static inline bool klp_is_object_loaded(struct klp_object *obj) 12static inline bool klp_is_object_loaded(struct klp_object *obj)
11{ 13{
12 return !obj->name || obj->mod; 14 return !obj->name || obj->mod;
diff --git a/kernel/livepatch/transition.c b/kernel/livepatch/transition.c
index a4c921364003..c9917a24b3a4 100644
--- a/kernel/livepatch/transition.c
+++ b/kernel/livepatch/transition.c
@@ -134,13 +134,6 @@ static void klp_complete_transition(void)
134 pr_notice("'%s': %s complete\n", klp_transition_patch->mod->name, 134 pr_notice("'%s': %s complete\n", klp_transition_patch->mod->name,
135 klp_target_state == KLP_PATCHED ? "patching" : "unpatching"); 135 klp_target_state == KLP_PATCHED ? "patching" : "unpatching");
136 136
137 /*
138 * patch->forced set implies unbounded increase of module's ref count if
139 * the module is disabled/enabled in a loop.
140 */
141 if (!klp_transition_patch->forced && klp_target_state == KLP_UNPATCHED)
142 module_put(klp_transition_patch->mod);
143
144 klp_target_state = KLP_UNDEFINED; 137 klp_target_state = KLP_UNDEFINED;
145 klp_transition_patch = NULL; 138 klp_transition_patch = NULL;
146} 139}
@@ -357,6 +350,7 @@ void klp_try_complete_transition(void)
357{ 350{
358 unsigned int cpu; 351 unsigned int cpu;
359 struct task_struct *g, *task; 352 struct task_struct *g, *task;
353 struct klp_patch *patch;
360 bool complete = true; 354 bool complete = true;
361 355
362 WARN_ON_ONCE(klp_target_state == KLP_UNDEFINED); 356 WARN_ON_ONCE(klp_target_state == KLP_UNDEFINED);
@@ -405,7 +399,18 @@ void klp_try_complete_transition(void)
405 } 399 }
406 400
407 /* we're done, now cleanup the data structures */ 401 /* we're done, now cleanup the data structures */
402 patch = klp_transition_patch;
408 klp_complete_transition(); 403 klp_complete_transition();
404
405 /*
406 * It would make more sense to free the patch in
407 * klp_complete_transition() but it is called also
408 * from klp_cancel_transition().
409 */
410 if (!patch->enabled) {
411 klp_free_patch_start(patch);
412 schedule_work(&patch->free_work);
413 }
409} 414}
410 415
411/* 416/*
diff --git a/samples/livepatch/livepatch-callbacks-demo.c b/samples/livepatch/livepatch-callbacks-demo.c
index 72f9e6d1387b..62d97953ad02 100644
--- a/samples/livepatch/livepatch-callbacks-demo.c
+++ b/samples/livepatch/livepatch-callbacks-demo.c
@@ -195,22 +195,11 @@ static struct klp_patch patch = {
195 195
196static int livepatch_callbacks_demo_init(void) 196static int livepatch_callbacks_demo_init(void)
197{ 197{
198 int ret; 198 return klp_enable_patch(&patch);
199
200 ret = klp_register_patch(&patch);
201 if (ret)
202 return ret;
203 ret = klp_enable_patch(&patch);
204 if (ret) {
205 WARN_ON(klp_unregister_patch(&patch));
206 return ret;
207 }
208 return 0;
209} 199}
210 200
211static void livepatch_callbacks_demo_exit(void) 201static void livepatch_callbacks_demo_exit(void)
212{ 202{
213 WARN_ON(klp_unregister_patch(&patch));
214} 203}
215 204
216module_init(livepatch_callbacks_demo_init); 205module_init(livepatch_callbacks_demo_init);
diff --git a/samples/livepatch/livepatch-sample.c b/samples/livepatch/livepatch-sample.c
index 2d554dd930e2..01c9cf003ca2 100644
--- a/samples/livepatch/livepatch-sample.c
+++ b/samples/livepatch/livepatch-sample.c
@@ -69,22 +69,11 @@ static struct klp_patch patch = {
69 69
70static int livepatch_init(void) 70static int livepatch_init(void)
71{ 71{
72 int ret; 72 return klp_enable_patch(&patch);
73
74 ret = klp_register_patch(&patch);
75 if (ret)
76 return ret;
77 ret = klp_enable_patch(&patch);
78 if (ret) {
79 WARN_ON(klp_unregister_patch(&patch));
80 return ret;
81 }
82 return 0;
83} 73}
84 74
85static void livepatch_exit(void) 75static void livepatch_exit(void)
86{ 76{
87 WARN_ON(klp_unregister_patch(&patch));
88} 77}
89 78
90module_init(livepatch_init); 79module_init(livepatch_init);
diff --git a/samples/livepatch/livepatch-shadow-fix1.c b/samples/livepatch/livepatch-shadow-fix1.c
index e8f1bd6b29b1..a5a5cac21d4d 100644
--- a/samples/livepatch/livepatch-shadow-fix1.c
+++ b/samples/livepatch/livepatch-shadow-fix1.c
@@ -157,25 +157,13 @@ static struct klp_patch patch = {
157 157
158static int livepatch_shadow_fix1_init(void) 158static int livepatch_shadow_fix1_init(void)
159{ 159{
160 int ret; 160 return klp_enable_patch(&patch);
161
162 ret = klp_register_patch(&patch);
163 if (ret)
164 return ret;
165 ret = klp_enable_patch(&patch);
166 if (ret) {
167 WARN_ON(klp_unregister_patch(&patch));
168 return ret;
169 }
170 return 0;
171} 161}
172 162
173static void livepatch_shadow_fix1_exit(void) 163static void livepatch_shadow_fix1_exit(void)
174{ 164{
175 /* Cleanup any existing SV_LEAK shadow variables */ 165 /* Cleanup any existing SV_LEAK shadow variables */
176 klp_shadow_free_all(SV_LEAK, livepatch_fix1_dummy_leak_dtor); 166 klp_shadow_free_all(SV_LEAK, livepatch_fix1_dummy_leak_dtor);
177
178 WARN_ON(klp_unregister_patch(&patch));
179} 167}
180 168
181module_init(livepatch_shadow_fix1_init); 169module_init(livepatch_shadow_fix1_init);
diff --git a/samples/livepatch/livepatch-shadow-fix2.c b/samples/livepatch/livepatch-shadow-fix2.c
index b34c7bf83356..52de947b5526 100644
--- a/samples/livepatch/livepatch-shadow-fix2.c
+++ b/samples/livepatch/livepatch-shadow-fix2.c
@@ -129,25 +129,13 @@ static struct klp_patch patch = {
129 129
130static int livepatch_shadow_fix2_init(void) 130static int livepatch_shadow_fix2_init(void)
131{ 131{
132 int ret; 132 return klp_enable_patch(&patch);
133
134 ret = klp_register_patch(&patch);
135 if (ret)
136 return ret;
137 ret = klp_enable_patch(&patch);
138 if (ret) {
139 WARN_ON(klp_unregister_patch(&patch));
140 return ret;
141 }
142 return 0;
143} 133}
144 134
145static void livepatch_shadow_fix2_exit(void) 135static void livepatch_shadow_fix2_exit(void)
146{ 136{
147 /* Cleanup any existing SV_COUNTER shadow variables */ 137 /* Cleanup any existing SV_COUNTER shadow variables */
148 klp_shadow_free_all(SV_COUNTER, NULL); 138 klp_shadow_free_all(SV_COUNTER, NULL);
149
150 WARN_ON(klp_unregister_patch(&patch));
151} 139}
152 140
153module_init(livepatch_shadow_fix2_init); 141module_init(livepatch_shadow_fix2_init);