You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-06 - 00:23
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Fri Dec 05, 2025 12:44:35)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
522921110,7sleep15231-21xargs09:59:081
522921110,7sleep15231-21xargs09:59:081
1713199966,80cyclictest10464-21ssh11:08:571
17131999660,17cyclictest181rcu_preempt08:31:481
17127999455,7cyclictest181rcu_preempt10:44:310
17127999455,7cyclictest181rcu_preempt10:44:310
17131999352,16cyclictest181rcu_preempt10:43:481
17131999352,16cyclictest181rcu_preempt10:43:481
17131999251,10cyclictest181rcu_preempt11:27:341
17127999069,12cyclictest0-21swapper/011:48:320
17127999051,28cyclictest0-21swapper/009:23:410
17127999051,28cyclictest0-21swapper/009:23:410
1698329013,47sleep217028-21awk07:08:302
17131998944,12cyclictest181rcu_preempt11:49:291
17131998939,7cyclictest181rcu_preempt09:11:581
17137998864,10cyclictest0-21swapper/211:47:352
1713199888,58cyclictest15983-21apt-get11:18:541
17131998746,13cyclictest181rcu_preempt09:48:551
17131998733,30cyclictest0-21swapper/111:17:431
17131998733,30cyclictest0-21swapper/111:17:431
17127998768,10cyclictest0-21swapper/007:08:340
17127998765,13cyclictest0-21swapper/007:38:500
17137998667,10cyclictest0-21swapper/209:38:322
17137998665,11cyclictest0-21swapper/209:08:482
17137998665,10cyclictest0-21swapper/208:38:332
17137998664,10cyclictest0-21swapper/208:18:332
17137998664,10cyclictest0-21swapper/208:18:332
17131998650,13cyclictest181rcu_preempt10:23:581
17131998650,13cyclictest181rcu_preempt10:23:581
17131998640,8cyclictest181rcu_preempt07:23:321
17127998615,35cyclictest22149-21taskset09:32:430
17137998561,16cyclictest292-21dbus-daemon10:58:292
17131998537,16cyclictest181rcu_preempt12:37:501
1713199853,44cyclictest8542-21ssh09:08:321
17127998565,14cyclictest2637-21sh08:38:300
17127998553,21cyclictest23510-21latency_hist07:38:340
17137998466,9cyclictest0-21swapper/207:23:322
17137998463,10cyclictest0-21swapper/210:23:522
17137998463,10cyclictest0-21swapper/210:23:522
17131998458,10cyclictest181rcu_preempt12:23:591
17131998442,8cyclictest181rcu_preempt09:43:301
17131998442,8cyclictest181rcu_preempt09:43:301
17131998428,14cyclictest181rcu_preempt10:12:201
17131998428,14cyclictest181rcu_preempt10:12:201
17137998361,10cyclictest0-21swapper/208:43:322
17131998351,11cyclictest181rcu_preempt11:39:061
17127998347,23cyclictest5878-21cut11:58:490
17137998264,8cyclictest0-21swapper/211:58:512
17131998249,17cyclictest181rcu_preempt09:13:531
17131998243,14cyclictest181rcu_preempt09:43:531
17127998258,16cyclictest5335-21systemctl11:58:290
17131998136,13cyclictest181rcu_preempt12:17:081
17131998124,10cyclictest0-21swapper/109:54:541
17127998166,8cyclictest0-21swapper/008:28:330
17137998064,12cyclictest17118-21cyclictest11:09:402
17137998059,11cyclictest0-21swapper/211:38:502
17137998057,17cyclictest1-21systemd08:20:472
17137998057,17cyclictest1-21systemd08:20:472
17131998048,13cyclictest181rcu_preempt12:18:521
17131998048,13cyclictest181rcu_preempt12:18:521
17131998045,13cyclictest181rcu_preempt12:29:441
17127998062,9cyclictest0-21swapper/008:23:320
17127998062,9cyclictest0-21swapper/008:23:320
17127998060,10cyclictest0-21swapper/008:43:330
17127998051,13cyclictest181rcu_preempt09:34:110
26352790,4sleep12639-21modprobe08:38:301
17137997955,15cyclictest207-21systemd-journal11:35:492
17137997951,19cyclictest1415-21switchtime11:49:482
17131997953,9cyclictest11286-21ssh12:08:541
17131997953,9cyclictest11286-21ssh12:08:541
17131997947,9cyclictest181rcu_preempt10:06:411
17127997965,7cyclictest0-21swapper/011:19:250
1712799793,30cyclictest19736-21fschecks_time12:23:530
17137997863,7cyclictest0-21swapper/207:58:332
17137997862,8cyclictest0-21swapper/211:54:032
17137997861,9cyclictest0-21swapper/212:28:532
17137997859,14cyclictest8448-21diskmemload10:59:062
17137997859,14cyclictest8448-21diskmemload10:59:062
17137997857,11cyclictest0-21swapper/207:48:332
17131997847,13cyclictest181rcu_preempt12:03:181
17131997846,23cyclictest7415-21sh11:03:321
17131997846,23cyclictest7415-21sh11:03:321
17131997842,8cyclictest181rcu_preempt10:38:221
17131997837,12cyclictest0-21swapper/110:40:481
17131997837,12cyclictest0-21swapper/110:40:481
17131997829,13cyclictest0-21swapper/111:44:051
17127997865,7cyclictest0-21swapper/009:49:050
17127997863,11cyclictest17118-21cyclictest10:38:540
17127997863,11cyclictest17118-21cyclictest10:38:540
17127997858,9cyclictest0-21swapper/012:14:070
17127997817,32cyclictest0-21swapper/011:52:100
17137997759,12cyclictest1966-21sh10:52:482
17137997753,12cyclictest0-21swapper/207:13:342
17137997753,12cyclictest0-21swapper/207:13:342
17131997745,8cyclictest181rcu_preempt11:53:581
17131997743,13cyclictest181rcu_preempt12:04:071
17131997742,13cyclictest181rcu_preempt07:53:511
17131997739,17cyclictest181rcu_preempt09:18:501
17131997735,17cyclictest181rcu_preempt09:24:371
17131997735,17cyclictest181rcu_preempt09:24:371
17131997731,11cyclictest181rcu_preempt08:54:021
17131997723,17cyclictest0-21swapper/110:58:331
17131997722,31cyclictest0-21swapper/107:38:301
17131997720,31cyclictest0-21swapper/108:23:591
17127997765,7cyclictest0-21swapper/012:18:550
17127997765,7cyclictest0-21swapper/012:18:550
17127997759,8cyclictest0-21swapper/012:08:560
17127997759,8cyclictest0-21swapper/012:08:560
17127997747,21cyclictest0-21swapper/010:34:090
17137997662,8cyclictest0-21swapper/210:28:522
17137997643,10cyclictest181rcu_preempt10:15:292
17127997659,8cyclictest0-21swapper/010:15:040
17127997658,13cyclictest8376-21cstates12:03:490
17137997560,11cyclictest17118-21cyclictest10:19:072
17137997559,9cyclictest0-21swapper/207:53:332
17137997553,12cyclictest0-21swapper/210:38:332
17137997552,16cyclictest314-21in:imuxsock09:17:472
17137997548,19cyclictest1-21systemd11:06:482
17137997544,22cyclictest1042-21cron09:53:292
17131997554,14cyclictest10045-21latency_hist11:08:321
17131997551,9cyclictest181rcu_preempt10:22:281
17131997551,8cyclictest13552-21ssh10:14:301
17131997550,11cyclictest181rcu_preempt10:29:101
17131997540,13cyclictest181rcu_preempt08:18:331
17131997540,13cyclictest181rcu_preempt08:18:331
17131997533,9cyclictest17118-21cyclictest07:13:361
17131997531,12cyclictest181rcu_preempt07:45:581
17127997561,8cyclictest0-21swapper/010:12:020
17127997561,8cyclictest0-21swapper/010:12:020
17127997551,17cyclictest0-21swapper/009:08:590
17127997546,7cyclictest0-21swapper/011:23:550
17137997459,8cyclictest0-21swapper/210:38:492
17137997459,8cyclictest0-21swapper/210:38:492
17137997458,7cyclictest0-21swapper/208:28:332
17137997451,16cyclictest9110-21timerandwakeup12:04:072
17137997446,9cyclictest181rcu_preempt09:39:262
17137997446,9cyclictest181rcu_preempt09:39:262
17137997412,34cyclictest39-21ksoftirqd/210:04:452
17131997443,7cyclictest181rcu_preempt07:09:051
17131997443,7cyclictest181rcu_preempt07:09:051
17131997442,13cyclictest181rcu_preempt11:30:571
17131997442,13cyclictest181rcu_preempt11:30:571
17131997440,12cyclictest181rcu_preempt09:28:521
17131997422,23cyclictest0-21swapper/107:24:061
17127997457,12cyclictest8448-21diskmemload09:42:060
17127997457,12cyclictest8448-21diskmemload09:42:060
17127997449,18cyclictest16715-21latency_hist09:23:330
17127997441,27cyclictest15593-21/usr/sbin/munin10:18:500
17127997416,29cyclictest8448-21diskmemload09:46:270
17137997354,8cyclictest0-21swapper/209:28:332
17137997354,8cyclictest0-21swapper/209:28:332
17137997351,12cyclictest4016-21cron09:58:292
17137997340,23cyclictest4485-21kworker/u19:0+nfsiod12:10:582
17137997340,23cyclictest4485-21kworker/u19:0+nfsiod12:10:582
17131997342,13cyclictest181rcu_preempt10:49:211
17131997336,7cyclictest181rcu_preempt09:03:321
17131997336,7cyclictest181rcu_preempt09:03:321
17131997333,13cyclictest181rcu_preempt11:33:461
17131997330,7cyclictest181rcu_preempt09:37:571
17127997360,9cyclictest17118-21cyclictest10:59:170
17127997360,9cyclictest17118-21cyclictest10:59:170
17127997355,10cyclictest0-21swapper/011:34:020
17127997354,14cyclictest691-21cron07:18:300
17127997352,15cyclictest8448-21diskmemload09:56:010
17145997232,28cyclictest13946-21chrt09:18:293
17137997254,13cyclictest12322-21apt-key10:13:332
17137997254,13cyclictest12322-21apt-key10:13:332
1713199722,29cyclictest207-21systemd-journal08:03:301
17127997259,9cyclictest17118-21cyclictest10:26:460
17127997259,9cyclictest17118-21cyclictest10:26:460
17127997253,7cyclictest181rcu_preempt10:30:330
1712799722,34cyclictest15-21ksoftirqd/008:11:180
17137997154,10cyclictest8448-21diskmemload09:18:512
17137997153,9cyclictest0-21swapper/210:44:032
17137997153,9cyclictest0-21swapper/210:44:032
17137997150,9cyclictest0-21swapper/210:03:332
17137997150,9cyclictest0-21swapper/210:03:332
1713799712,36cyclictest39-21ksoftirqd/212:26:322
17127997159,7cyclictest0-21swapper/010:57:260
17127997153,9cyclictest0-21swapper/010:48:410
17127997153,9cyclictest0-21swapper/007:23:300
1712799714,34cyclictest15-21ksoftirqd/012:37:520
17127997142,23cyclictest0-21swapper/011:13:580
17127997142,23cyclictest0-21swapper/011:13:580
17137997054,9cyclictest0-21swapper/211:28:542
17137997054,9cyclictest0-21swapper/211:28:542
17131997039,9cyclictest181rcu_preempt08:45:421
17131997029,10cyclictest181rcu_preempt07:32:321
17131997026,19cyclictest0-21swapper/108:38:571
17131997016,8cyclictest0-21swapper/108:09:061
17131997010,31cyclictest31-21ksoftirqd/107:53:341
17127997056,10cyclictest17118-21cyclictest09:59:540
17127997056,10cyclictest17118-21cyclictest09:59:540
17127997048,15cyclictest17118-21cyclictest07:53:460
1712799702,35cyclictest15-21ksoftirqd/007:48:480
1714599693,51cyclictest32263-21taskset10:48:503
17145996932,21cyclictest8759-21chrt12:03:583
17137996954,8cyclictest0-21swapper/212:19:072
17137996954,8cyclictest0-21swapper/212:19:072
17137996954,8cyclictest0-21swapper/211:24:052
17137996951,9cyclictest0-21swapper/209:08:322
17137996950,10cyclictest0-21swapper/208:53:492
17131996942,12cyclictest31-21ksoftirqd/108:52:591
17127996952,8cyclictest0-21swapper/011:38:540
17127996939,26cyclictest18430-21kworker/u18:3-rpciod09:17:350
17127996925,8cyclictest161ktimers/011:08:390
1714599683,52cyclictest19830-21taskset11:24:363
17145996832,22cyclictest8295-21chrt11:04:023
17145996831,24cyclictest13164-21taskset12:12:453
17145996831,24cyclictest13164-21taskset12:12:453
17145996831,23cyclictest7166-21chrt12:00:303
17137996854,8cyclictest0-21swapper/208:30:472
17137996853,8cyclictest0-21swapper/211:13:482
17137996853,8cyclictest0-21swapper/211:13:482
1713799682,34cyclictest39-21ksoftirqd/212:15:532
17127996852,10cyclictest316-21rs:main0
1712799682,33cyclictest15-21ksoftirqd/010:05:170
1714599673,46cyclictest16859-21taskset09:23:323
17137996752,8cyclictest0-21swapper/209:29:042
17137996749,9cyclictest0-21swapper/208:08:562
17137996747,10cyclictest0-21swapper/208:48:332
17137996731,7cyclictest181rcu_preempt09:46:582
1713799672,31cyclictest39-21ksoftirqd/212:33:472
1713199672,32cyclictest31-21ksoftirqd/108:20:411
1713199672,32cyclictest31-21ksoftirqd/108:20:411
17127996735,24cyclictest17899-21wc07:09:000
17127996735,24cyclictest17899-21wc07:09:000
1712799672,28cyclictest15-21ksoftirqd/008:05:100
17127996720,18cyclictest15-21ksoftirqd/012:32:510
17145996632,20cyclictest25238-21chrt10:35:293
17145996632,20cyclictest10787-21chrt10:09:073
17145996632,20cyclictest10787-21chrt10:09:073
17145996631,21cyclictest24488-21chrt11:33:523
17145996630,22cyclictest22812-21chrt11:30:233
17145996630,22cyclictest22812-21chrt11:30:233
17145996629,24cyclictest14100-21taskset10:14:503
17145996629,22cyclictest7549-21taskset11:03:333
17145996629,22cyclictest7549-21taskset11:03:333
17145996628,24cyclictest5791-21taskset10:00:083
17145996628,24cyclictest5791-21taskset10:00:083
17137996652,7cyclictest0-21swapper/207:13:362
17137996651,8cyclictest0-21swapper/211:19:142
17137996647,9cyclictest0-21swapper/209:03:312
17137996647,9cyclictest0-21swapper/209:03:312
1713799662,27cyclictest39-21ksoftirqd/207:38:472
17131996634,11cyclictest181rcu_preempt07:38:451
17127996653,7cyclictest0-21swapper/009:08:320
1714599654,39cyclictest0-21swapper/311:48:553
17145996533,20cyclictest9221-21chrt09:08:533
17145996531,21cyclictest27701-21chrt10:39:303
17145996531,21cyclictest27701-21chrt10:39:303
17145996530,22cyclictest26957-21taskset07:53:583
17145996529,23cyclictest15628-21chrt10:18:433
17145996529,21cyclictest27393-21taskset11:38:543
17137996547,8cyclictest0-21swapper/208:03:332
17137996544,15cyclictest1-21systemd07:25:482
17131996535,8cyclictest181rcu_preempt08:08:311
17127996549,7cyclictest0-21swapper/007:28:320
17145996434,19cyclictest31993-21chrt08:23:293
17145996434,19cyclictest31993-21chrt08:23:293
17145996433,20cyclictest22876-21chrt09:33:463
17145996432,20cyclictest19849-21chrt12:23:553
17145996432,20cyclictest18327-21chrt11:23:303
17145996432,20cyclictest1670-21taskset09:53:553
17145996432,19cyclictest7597-21taskset10:03:563
17145996432,19cyclictest22296-21chrt12:28:463
17145996430,22cyclictest4153-21chrt08:42:083
17145996428,25cyclictest4729-21chrt08:46:263
17145996427,25cyclictest22238-21taskset07:28:353
17145996427,25cyclictest20132-21taskset07:20:213
17137996411,25cyclictest39-21ksoftirqd/207:31:572
1712799647,27cyclictest15-21ksoftirqd/007:47:530
17127996447,10cyclictest316-21rs:main0
1712799644,30cyclictest15-21ksoftirqd/011:28:480
1712799644,30cyclictest15-21ksoftirqd/011:28:480
1712799644,29cyclictest15-21ksoftirqd/008:58:490
1712799644,29cyclictest15-21ksoftirqd/008:58:490
17145996332,20cyclictest30284-21chrt10:44:083
17145996332,20cyclictest30284-21chrt10:44:083
17145996331,21cyclictest30327-21chrt09:47:073
17145996331,20cyclictest26258-21chrt12:34:213
17145996331,20cyclictest25541-21chrt09:38:443
17145996331,20cyclictest25541-21chrt09:38:443
17145996330,21cyclictest32107-21chrt11:47:533
17145996330,21cyclictest32029-21chrt09:49:053
17145996330,21cyclictest19602-21chrt07:17:013
17145996330,20cyclictest4155-21chrt10:56:323
17145996330,20cyclictest14431-21chrt11:15:083
17145996330,20cyclictest14431-21chrt11:15:083
17145996327,25cyclictest21068-21taskset10:28:403
17145996324,26cyclictest18564-21taskset07:12:273
17145996324,26cyclictest18564-21taskset07:12:273
17127996347,10cyclictest20422-21kworker/u18:2+events_unbound08:33:320
17127996336,10cyclictest58-21kcompactd008:16:570
17127996336,10cyclictest58-21kcompactd008:16:570
17145996231,20cyclictest25553-21chrt07:48:313
17145996230,22cyclictest13749-21taskset12:13:423
17145996229,21cyclictest17323-21chrt09:23:533
17145996229,21cyclictest17323-21chrt09:23:533
17145996229,19cyclictest21678-21chrt09:31:143
17145996227,22cyclictest1846-21chrt08:31:403
1713799622,24cyclictest39-21ksoftirqd/207:35:482
17131996241,7cyclictest181rcu_preempt07:08:341
17145996127,22cyclictest28318-21taskset08:02:123
17145996127,21cyclictest23560-21chrt07:38:433
17145996126,24cyclictest12519-21chrt11:12:493
17145996126,22cyclictest22889-21taskset07:33:493
17145996125,24cyclictest8323-21taskset09:07:073
17145996121,27cyclictest30969-21taskset08:14:183
17145996121,27cyclictest30969-21taskset08:14:183
17137996130,21cyclictest5531-21modprobe08:53:302
17127996144,8cyclictest0-21swapper/011:03:460
1712799613,25cyclictest15-21ksoftirqd/008:53:460
17145996029,20cyclictest25768-21chrt07:48:353
17145996029,19cyclictest19892-21chrt10:25:403
17145996029,19cyclictest19892-21chrt10:25:403
17145996028,21cyclictest16643-21chrt12:18:453
17145996028,21cyclictest16643-21chrt12:18:453
17145996021,26cyclictest28711-21taskset08:04:023
17145995930,19cyclictest7517-21chrt08:59:503
17145995930,19cyclictest7517-21chrt08:59:503
17145995929,19cyclictest6950-21chrt08:56:093
17145995928,20cyclictest2345-21chrt08:34:513
17145995928,19cyclictest4231-21chrt11:54:463
17145995927,20cyclictest808-21chrt08:24:313
17145995926,20cyclictest20665-21chrt07:23:453
17145995925,21cyclictest30338-21chrt08:09:203
1712799583,26cyclictest15-21ksoftirqd/008:48:520
17145995724,21cyclictest5149-21chrt08:48:403
1713799562,22cyclictest39-21ksoftirqd/208:04:252
1712799563,30cyclictest15-21ksoftirqd/007:59:030
1714599473,32cyclictest17192-21chrt07:08:343
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional