You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-29 - 12:28
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Sun Mar 29, 2026 00:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5164991223,47cyclictest5354-21apt-get19:04:300
5164991223,47cyclictest5354-21apt-get19:04:300
5164991133,35cyclictest5155-21cyclictest19:04:210
51649911033,69cyclictest0-21swapper/023:59:350
51649910928,7cyclictest0-21swapper/023:54:190
5164991083,52cyclictest5284-21apt-get00:30:040
4041210711,69sleep14051-21/usr/sbin/munin18:59:471
51649910543,55cyclictest352-21runrttasks19:30:020
5164991053,33cyclictest25677-21/usr/sbin/munin23:14:470
5164991053,32cyclictest28492-21latency_hist20:14:200
5164991052,25cyclictest16362-21/usr/sbin/munin22:04:410
51649910520,25cyclictest9511-21latency_hist21:54:170
5164991043,27cyclictest31562-21find22:29:510
51649910428,20cyclictest24603-21cstates21:24:350
51649910427,72cyclictest0-21swapper/000:09:530
5164991042,37cyclictest3872-21/usr/sbin/munin21:44:450
5164991042,37cyclictest3872-21/usr/sbin/munin21:44:450
51649910337,59cyclictest28917-21fschecks_time00:14:410
5164991033,31cyclictest21743-21grep21:19:380
5164991033,31cyclictest21743-21grep21:19:380
5164991033,22cyclictest352-21runrttasks22:22:520
51649910236,60cyclictest544-21cron21:34:220
5164991023,34cyclictest1166-21tune2fs21:39:390
5164991023,28cyclictest14343-21unixbench_singl22:54:550
5164991023,24cyclictest31686-21ntp_states23:24:480
51649910229,9cyclictest0-21swapper/022:34:430
51649910227,9cyclictest0-21swapper/021:15:370
51649910136,59cyclictest27429-21/usr/sbin/munin21:29:370
51649910134,57cyclictest25346-21latency_hist20:04:210
5164991012,28cyclictest17302-21/usr/sbin/munin19:39:520
5164991012,28cyclictest17302-21/usr/sbin/munin19:39:520
51889910042,11cyclictest0-21swapper/219:09:202
51889910042,11cyclictest0-21swapper/219:09:202
51649910035,57cyclictest0-21swapper/020:24:530
5164991003,36cyclictest6465-21cut20:44:490
5164991003,36cyclictest6465-21cut20:44:490
5164991003,36cyclictest352-21runrttasks23:33:270
5164991003,36cyclictest352-21runrttasks23:33:270
5164991003,34cyclictest14017-21cat19:29:210
5164991003,31cyclictest12088-21cron21:04:170
5164991003,31cyclictest12088-21cron21:04:170
5164991003,30cyclictest11108-21fschecks_count19:19:570
51649910025,11cyclictest0-21swapper/023:34:380
5164991002,32cyclictest4096-21ssh00:27:320
51649910022,11cyclictest0-21swapper/020:49:530
5164999939,53cyclictest5970-21sh22:41:230
5164999939,53cyclictest5970-21sh22:41:230
516499993,31cyclictest352-21runrttasks23:43:110
516499993,31cyclictest352-21runrttasks23:43:110
516499993,31cyclictest2617-21/usr/sbin/munin20:34:520
516499992,34cyclictest25307-21ssh00:09:180
516499992,34cyclictest22413-21fschecks_time19:54:410
516499992,26cyclictest16816-21grep22:59:470
516499992,26cyclictest16816-21grep22:59:470
516499992,23cyclictest22698-21/usr/sbin/munin23:09:450
516499992,23cyclictest22698-21/usr/sbin/munin23:09:450
5164999910,26cyclictest12109-21diskmemload21:04:560
516499984,31cyclictest29967-21modprobe20:19:170
5164999839,53cyclictest20380-21latency_hist19:49:200
5164999839,53cyclictest20380-21latency_hist19:49:200
5164999835,8cyclictest18936-21rm23:58:080
516499983,31cyclictest1791-21timerandwakeup20:29:530
5164999829,7cyclictest0-21swapper/000:20:190
5164999815,24cyclictest15729-21fschecks_time21:09:410
5164999815,24cyclictest15729-21fschecks_time21:09:410
516499973,36cyclictest13442-21/usr/sbin/munin21:59:420
516499973,33cyclictest7554-21fschecks_count22:44:390
5164999733,11cyclictest9307-21fschecks_count20:54:390
516499973,29cyclictest20639-21cstates19:49:370
516499973,28cyclictest10333-21cstates21:54:560
516499972,29cyclictest19170-21acpi23:04:270
5164999719,22cyclictest1-21systemd20:44:190
516499963,32cyclictest352-21runrttasks22:19:150
516499963,32cyclictest13344-21latency_hist23:49:200
516499962,35cyclictest17241-21latency_hist19:39:210
516499962,31cyclictest28641-21rm22:24:510
5164999535,53cyclictest0-21swapper/022:14:200
5164999525,13cyclictest0-21swapper/020:19:390
516499952,36cyclictest9439-21latency19:14:430
516499952,36cyclictest9439-21latency19:14:430
516499952,33cyclictest25567-21cut20:04:400
516499943,25cyclictest352-21runrttasks23:20:580
516499943,25cyclictest352-21runrttasks23:20:580
516499933,31cyclictest7374-21/usr/sbin/munin19:09:460
516499932,25cyclictest10448-21expr22:49:360
516499932,25cyclictest10448-21expr22:49:360
5188999148,11cyclictest5155-21cyclictest23:19:552
5188999148,11cyclictest5155-21cyclictest23:19:552
5180999068,15cyclictest4031-21sh22:39:171
5180999068,15cyclictest12109-21diskmemload23:59:401
5180998967,16cyclictest352-21runrttasks23:49:521
371228913,50sleep23763-21sort18:59:362
5188998868,13cyclictest17225-21latency_hist19:39:212
5188998753,21cyclictest0-21swapper/223:54:402
518899874,51cyclictest5284-21apt-get00:29:362
518899872,52cyclictest1693-21ls23:29:352
518899872,52cyclictest1693-21ls23:29:352
518099876,57cyclictest31-21ksoftirqd/120:55:171
5188998646,15cyclictest20918-21ssh22:12:172
5180998615,62cyclictest301ktimers/123:29:171
518899857,60cyclictest18112-21apt-get21:14:332
5188998556,19cyclictest18479-21latency_hist21:14:222
5188998556,19cyclictest18479-21latency_hist21:14:222
5180998554,20cyclictest11120-21sort19:19:571
5188998463,15cyclictest12109-21diskmemload21:19:402
5188998463,15cyclictest12109-21diskmemload21:19:402
5188998461,17cyclictest11849-21sh21:56:482
5188998446,8cyclictest181rcu_preempt23:04:282
5180998465,12cyclictest24845-21kworker/u17:1-nfsiod23:46:461
518099843,54cyclictest31-21ksoftirqd/119:34:551
5188998343,15cyclictest181rcu_preempt22:53:342
5188998343,15cyclictest181rcu_preempt22:53:342
518099838,43cyclictest31-21ksoftirqd/119:04:471
518099838,43cyclictest31-21ksoftirqd/119:04:471
5180998346,11cyclictest181rcu_preempt21:52:021
5180998314,38cyclictest31-21ksoftirqd/119:54:531
518099826,42cyclictest31-21ksoftirqd/121:27:471
5180998260,14cyclictest2952-21systemd-journal22:55:381
5180998236,14cyclictest181rcu_preempt23:05:091
5188998154,8cyclictest181rcu_preempt00:04:392
5180998164,11cyclictest5608-21grep23:34:531
5180998150,9cyclictest181rcu_preempt20:49:201
5180998150,9cyclictest181rcu_preempt20:49:201
518099812,26cyclictest19221-21cut22:09:391
518099811,46cyclictest31-21ksoftirqd/123:40:081
518099811,46cyclictest31-21ksoftirqd/123:40:081
5180998063,11cyclictest5154-21unixbench_multi20:39:541
5180998052,10cyclictest181rcu_preempt21:32:091
5180998048,13cyclictest181rcu_preempt20:20:201
5180997953,11cyclictest181rcu_preempt21:22:371
5180997953,11cyclictest181rcu_preempt21:22:371
5180997943,13cyclictest181rcu_preempt20:27:071
5180997939,13cyclictest181rcu_preempt21:56:221
518099793,47cyclictest31-21ksoftirqd/122:31:321
518099791,35cyclictest31-21ksoftirqd/120:34:531
5188997856,12cyclictest0-21swapper/200:24:552
5188997855,15cyclictest28341-21sh20:14:172
5188997832,34cyclictest1728-21cat20:29:522
5180997862,10cyclictest10627-21idleruntime22:49:421
5180997862,10cyclictest10627-21idleruntime22:49:421
5180997856,15cyclictest7974-21ssh00:34:181
518099785,26cyclictest31-21ksoftirqd/119:09:281
5180997843,11cyclictest181rcu_preempt22:20:471
5180997841,12cyclictest181rcu_preempt20:08:571
5188997760,11cyclictest25446-21ssh22:19:452
5188997753,15cyclictest1-21systemd23:49:522
518099778,29cyclictest15987-21ssh22:04:281
5180997761,9cyclictest14816-21switchtime19:30:001
5180997759,11cyclictest2-21kthreadd20:14:171
5180997757,14cyclictest3374-21sh21:43:441
5180997735,28cyclictest12830-21cat21:04:431
5180997719,10cyclictest301ktimers/100:19:501
5180997713,33cyclictest31-21ksoftirqd/122:24:271
5188997659,11cyclictest26352-21sendmail_mailtr00:09:512
5188997655,15cyclictest300-21rs:main2
518099762,40cyclictest31-21ksoftirqd/100:09:451
5180997614,53cyclictest301ktimers/121:12:281
5180997614,53cyclictest301ktimers/121:12:281
5188997557,11cyclictest28053-21ntpq21:29:472
5188997545,11cyclictest181rcu_preempt22:59:282
5188997545,11cyclictest181rcu_preempt22:59:282
518099752,50cyclictest31-21ksoftirqd/122:47:171
518099752,38cyclictest31-21ksoftirqd/100:09:201
5188997457,11cyclictest2952-21systemd-journal23:36:382
5188997455,13cyclictest12109-21diskmemload23:59:422
5188997439,21cyclictest0-21swapper/200:14:352
5180997447,8cyclictest181rcu_preempt19:14:331
5180997447,8cyclictest181rcu_preempt19:14:331
5180997446,9cyclictest181rcu_preempt19:59:461
5180997440,10cyclictest181rcu_preempt20:14:231
519299737,36cyclictest0-21swapper/323:54:403
5188997357,10cyclictest10428-21systemd-run19:19:192
5188997357,10cyclictest10428-21systemd-run19:19:192
5188997356,10cyclictest12109-21diskmemload22:35:322
5188997349,15cyclictest352-21runrttasks20:14:482
5188997334,13cyclictest181rcu_preempt19:19:402
5188997330,10cyclictest181rcu_preempt20:09:212
518099739,39cyclictest31-21ksoftirqd/122:18:591
518099736,43cyclictest31-21ksoftirqd/123:09:451
518099736,43cyclictest31-21ksoftirqd/123:09:451
5180997348,16cyclictest30896-21latency21:34:431
5180997345,19cyclictest4660-21fschecks_time22:39:401
5180997345,19cyclictest4660-21fschecks_time22:39:401
5180997334,16cyclictest181rcu_preempt00:17:491
5180997310,36cyclictest31-21ksoftirqd/121:47:251
5180997310,36cyclictest31-21ksoftirqd/121:47:251
5188997251,8cyclictest39-21ksoftirqd/223:13:072
5188997251,8cyclictest39-21ksoftirqd/223:13:072
518099723,50cyclictest19212-21/usr/sbin/munin21:15:241
518099723,49cyclictest1621-21grep20:29:501
5180997234,14cyclictest181rcu_preempt19:24:491
518099722,40cyclictest31-21ksoftirqd/120:59:561
518099722,40cyclictest31-21ksoftirqd/120:59:561
5188997156,10cyclictest352-21runrttasks21:37:022
5188997135,11cyclictest5155-21cyclictest00:23:102
5180997141,8cyclictest181rcu_preempt23:34:111
5180997141,8cyclictest181rcu_preempt23:34:111
5180997137,12cyclictest181rcu_preempt19:44:071
5180997137,12cyclictest181rcu_preempt19:44:071
518099712,34cyclictest31-21ksoftirqd/123:54:261
5180997052,11cyclictest15697-21grep22:04:171
518099702,44cyclictest31-21ksoftirqd/123:01:311
518099702,44cyclictest31-21ksoftirqd/123:01:311
3612700,7sleep20-21swapper/223:26:482
5188996947,15cyclictest5155-21cyclictest20:44:462
5188996947,15cyclictest5155-21cyclictest20:44:462
5188996946,11cyclictest0-21swapper/221:24:552
5188996945,13cyclictest0-21swapper/223:42:392
5188996945,13cyclictest0-21swapper/223:42:392
518899693,46cyclictest7208-21idleruntime21:49:422
518899693,26cyclictest12460-21acpi19:24:272
518899686,40cyclictest27768-21ssh23:19:162
5180996852,10cyclictest25526-21if_eth023:14:431
5180996840,14cyclictest181rcu_preempt00:25:241
5180996835,15cyclictest181rcu_preempt19:50:351
5180996835,13cyclictest181rcu_preempt20:49:261
518099682,60cyclictest291rcuc/123:21:011
518099682,60cyclictest291rcuc/123:21:011
130072680,10sleep2371rcuc/221:59:232
5188996751,12cyclictest5155-21cyclictest21:04:382
518899673,45cyclictest352-21runrttasks22:26:252
518899673,23cyclictest8817-21apt20:54:172
518899672,47cyclictest4554-21/usr/sbin/munin22:39:392
518899672,47cyclictest4554-21/usr/sbin/munin22:39:392
518099677,28cyclictest31-21ksoftirqd/119:44:291
518099677,28cyclictest31-21ksoftirqd/119:44:291
5188996649,11cyclictest365-21ntpd22:44:502
5188996648,13cyclictest11622-21unixbench_singl23:45:062
5188996547,13cyclictest0-21swapper/221:49:202
5188996547,13cyclictest0-21swapper/221:49:202
5188996536,9cyclictest181rcu_preempt21:39:412
518899653,37cyclictest22849-21grep19:54:522
518899652,45cyclictest31726-21timerandwakeup22:29:532
5188996518,33cyclictest22009-21cpu22:14:322
5188996449,10cyclictest5155-21cyclictest19:09:532
5188996350,9cyclictest18732-21kworker/u18:3-rpciod22:08:192
5188996337,8cyclictest181rcu_preempt19:34:212
518899633,33cyclictest30305-21cpuspeed20:19:342
519299624,38cyclictest897-21taskset21:39:263
5188996246,9cyclictest0-21swapper/219:54:172
5188996239,7cyclictest181rcu_preempt22:54:392
5188996234,18cyclictest0-21swapper/219:39:392
5188996234,18cyclictest0-21swapper/219:39:392
518899622,43cyclictest9370-21fschecks_time20:54:392
518899622,38cyclictest5817-21date20:44:212
5188996147,9cyclictest58-21kcompactd020:35:072
5188996145,10cyclictest20254-21cron19:49:172
5188996145,10cyclictest20254-21cron19:49:172
5188996140,11cyclictest0-21swapper/220:59:572
5188996140,11cyclictest0-21swapper/220:59:572
518899592,18cyclictest23792-21sshd19:59:312
519299583,34cyclictest11387-21chrt19:20:033
519299574,35cyclictest27303-21chrt22:23:323
519299573,37cyclictest21856-21chrt00:03:003
519299573,37cyclictest17492-21chrt21:12:353
519299573,37cyclictest17492-21chrt21:12:353
519299572,37cyclictest32577-21chrt00:19:543
519299563,35cyclictest8746-21chrt19:14:173
519299562,39cyclictest8478-21taskset23:39:543
519299562,39cyclictest8478-21taskset23:39:543
519299562,36cyclictest16060-21chrt22:59:203
5192995615,25cyclictest0-21swapper/300:29:323
519299553,37cyclictest6075-21chrt22:41:413
519299553,37cyclictest6075-21chrt22:41:413
519299544,34cyclictest13860-21taskset23:49:433
519299543,39cyclictest22926-21chrt22:14:533
519299534,33cyclictest8938-21taskset22:46:233
519299533,38cyclictest19794-21chrt19:44:543
519299533,38cyclictest19794-21chrt19:44:543
519299533,37cyclictest30766-21chrt23:24:173
519299533,37cyclictest30766-21chrt23:24:173
519299533,36cyclictest13742-21chrt21:05:303
519299533,35cyclictest12874-21chrt21:59:203
519299533,34cyclictest3518-21chrt23:32:203
519299533,34cyclictest3518-21chrt23:32:203
519299533,33cyclictest31111-21chrt23:24:343
519299533,33cyclictest10394-21chrt22:49:353
519299533,33cyclictest10394-21chrt22:49:353
519299532,39cyclictest19545-21chrt22:09:463
442525315,23sleep30-21swapper/319:00:023
519299525,29cyclictest0-21swapper/321:14:333
519299523,36cyclictest21545-21chrt21:19:313
519299523,36cyclictest21545-21chrt21:19:313
519299523,34cyclictest19483-21chrt23:04:393
519299523,31cyclictest27733-21chrt20:09:533
519299523,31cyclictest14823-21chrt19:30:003
519299522,35cyclictest25995-21chrt21:26:043
519299513,35cyclictest6873-21chrt21:49:313
519299513,32cyclictest386-21chrt21:38:413
519299513,30cyclictest16235-21taskset22:04:393
519299512,36cyclictest16056-21chrt19:34:413
519299503,35cyclictest5146-21chrt20:39:543
519299503,34cyclictest29699-21chrt00:14:553
519299503,33cyclictest21926-21taskset19:54:183
519299502,34cyclictest31476-21chrt22:29:493
519299502,32cyclictest17328-21taskset22:59:573
519299502,32cyclictest17328-21taskset22:59:573
519299502,31cyclictest30573-21taskset20:19:413
519299502,31cyclictest1929-21chrt20:29:543
519299494,32cyclictest26462-21chrt23:15:343
519299493,34cyclictest11104-21chrt20:59:463
519299493,34cyclictest11104-21chrt20:59:463
519299483,33cyclictest26818-21taskset20:09:203
519299483,32cyclictest24489-21taskset23:13:273
519299483,32cyclictest24489-21taskset23:13:273
519299483,31cyclictest25401-21taskset00:09:203
519299483,29cyclictest917-21kworker/3:1+events22:34:323
519299482,36cyclictest12301-21chrt23:46:553
519299482,33cyclictest9402-21chrt20:54:403
519299482,32cyclictest29943-21chrt21:33:543
519299473,32cyclictest6126-21chrt23:36:133
519299473,31cyclictest10494-21chrt19:19:203
519299473,31cyclictest10494-21chrt19:19:203
519299466,27cyclictest27577-21taskset00:12:243
519299463,32cyclictest32576-21chrt20:24:513
519299463,31cyclictest3032-21chrt20:34:443
519299443,30cyclictest4522-21taskset21:44:493
519299443,30cyclictest4522-21taskset21:44:493
519299443,28cyclictest28892-21taskset20:14:393
519299442,29cyclictest7467-21chrt20:49:303
519299432,31cyclictest29272-21taskset22:25:433
519299432,28cyclictest6399-21taskset20:44:483
519299432,28cyclictest6399-21taskset20:44:483
519299422,28cyclictest13242-21taskset21:59:373
519299422,27cyclictest24558-21taskset19:59:533
519299422,27cyclictest22180-21taskset19:54:343
519299413,26cyclictest17327-21taskset19:39:313
519299413,26cyclictest17327-21taskset19:39:313
519299413,26cyclictest12481-21taskset19:24:293
519299412,29cyclictest2096-21taskset00:24:323
519299393,25cyclictest5698-21chrt19:04:423
519299393,25cyclictest5698-21chrt19:04:423
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional