You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-07 - 15:33
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Tue Jul 07, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2605821466,66sleep026089-21cut06:57:350
26129991243,88cyclictest9103-21seq07:43:191
26129991243,88cyclictest9103-21seq07:43:191
26129991163,84cyclictest28587-21chrt09:42:581
26129991163,84cyclictest28587-21chrt09:42:581
26129991128,86cyclictest8616-21apt-get10:57:501
261299910547,24cyclictest373-21lldpd12:02:331
26129991019,73cyclictest21101-21ssh12:13:051
2613199962,70cyclictest8616-21apt-get10:57:512
26129999563,12cyclictest181rcu_preempt09:48:081
26129999353,23cyclictest181rcu_preempt11:50:331
26129999352,8cyclictest181rcu_preempt12:05:421
26129999234,28cyclictest26115-21cyclictest11:56:061
26122999238,23cyclictest7402-21sh11:49:190
26131999066,17cyclictest1-21systemd09:31:052
26129999054,15cyclictest181rcu_preempt09:03:051
26129999054,15cyclictest181rcu_preempt09:03:051
26129999030,20cyclictest0-21swapper/111:39:211
26131998964,17cyclictest23852-21ssh11:22:552
26129998937,24cyclictest0-21swapper/110:18:071
26129998767,14cyclictest16332-21/usr/sbin/munin09:22:421
26129998767,14cyclictest16332-21/usr/sbin/munin09:22:421
26122998733,35cyclictest4677-21kworker/u17:0+xprtiod09:58:530
2603328712,36sleep20-21swapper/206:57:332
2613199862,57cyclictest8545-21apt-get07:42:522
2613199862,57cyclictest8545-21apt-get07:42:522
26129998664,8cyclictest181rcu_preempt10:43:061
26129998664,8cyclictest181rcu_preempt10:43:061
26129998659,11cyclictest181rcu_preempt08:43:591
26129998554,8cyclictest181rcu_preempt08:02:341
26129998550,8cyclictest181rcu_preempt08:22:531
26129998543,8cyclictest181rcu_preempt11:29:011
26129998543,8cyclictest181rcu_preempt11:29:011
26129998525,30cyclictest0-21swapper/109:38:271
26129998525,30cyclictest0-21swapper/109:38:271
26129998427,27cyclictest0-21swapper/107:06:401
26131998366,10cyclictest618-21kworker/u17:1-ext4-rsv-conversion08:57:342
26129998362,8cyclictest181rcu_preempt09:19:591
26129998355,14cyclictest181rcu_preempt07:32:491
26129998349,13cyclictest181rcu_preempt11:35:501
26129998348,8cyclictest181rcu_preempt10:01:561
26129998343,9cyclictest181rcu_preempt12:08:191
26122998336,25cyclictest15132-21kworker/u17:0+nfsiod10:13:380
2613199823,62cyclictest21016-21ssh12:12:522
26129998254,12cyclictest12449-21grep11:03:071
26129998254,12cyclictest12449-21grep11:03:071
26129998248,14cyclictest181rcu_preempt08:18:041
26129998246,8cyclictest181rcu_preempt11:43:081
26129998241,22cyclictest4677-21kworker/u17:0-writeback10:02:541
26129998241,22cyclictest4677-21kworker/u17:0-writeback10:02:541
26129998229,19cyclictest0-21swapper/110:25:001
26122998232,36cyclictest0-21swapper/008:57:350
26129998164,11cyclictest24063-21ssh12:17:491
26129998146,8cyclictest181rcu_preempt10:52:491
26129998144,19cyclictest181rcu_preempt09:34:521
26129998136,13cyclictest0-21swapper/108:02:541
26129998135,10cyclictest0-21swapper/110:38:041
26129998125,20cyclictest0-21swapper/109:29:561
26122998130,36cyclictest0-21swapper/008:28:000
26129998059,7cyclictest181rcu_preempt09:56:511
26129998059,7cyclictest181rcu_preempt09:56:511
26129998049,8cyclictest181rcu_preempt10:13:031
26129998047,8cyclictest181rcu_preempt08:12:501
26129998047,8cyclictest181rcu_preempt08:12:501
26129998044,9cyclictest181rcu_preempt10:28:251
26129998037,10cyclictest0-21swapper/108:37:341
26129998022,28cyclictest0-21swapper/110:10:011
26122998062,13cyclictest26115-21cyclictest07:39:180
26122998059,14cyclictest12232-21latency_hist07:52:340
26122998021,30cyclictest0-21swapper/011:03:280
26122998021,30cyclictest0-21swapper/011:03:280
26131997960,12cyclictest19294-21latency_hist09:27:352
2612999797,42cyclictest31-21ksoftirqd/109:15:041
26129997957,8cyclictest181rcu_preempt08:38:041
2612999794,43cyclictest31-21ksoftirqd/108:52:131
2612999794,43cyclictest31-21ksoftirqd/108:52:131
26129997919,35cyclictest0-21swapper/111:08:501
26122997963,10cyclictest28162-21latency_hist08:42:350
26122997924,40cyclictest26268-21kworker/u17:0+rpciod11:12:560
26122997924,22cyclictest0-21swapper/009:04:290
26122997924,22cyclictest0-21swapper/009:04:290
26122997921,35cyclictest0-21swapper/011:47:180
26122997921,34cyclictest0-21swapper/012:14:220
2482127911,26sleep10-21swapper/106:53:401
26131997817,24cyclictest4357-21seq11:44:112
26129997856,8cyclictest181rcu_preempt08:58:301
2612999785,41cyclictest31-21ksoftirqd/107:12:241
26129997848,8cyclictest181rcu_preempt11:21:471
26129997834,10cyclictest0-21swapper/110:48:071
26129997825,15cyclictest0-21swapper/108:31:431
26122997860,12cyclictest17460-21sshd11:58:060
26129997731,9cyclictest0-21swapper/107:23:001
26122997755,16cyclictest31730-21fschecks_count09:47:540
26122997755,15cyclictest30238-21latency_hist08:47:350
2612299773,40cyclictest24956-21modprobe09:37:300
26131997659,10cyclictest3262-21runrttasks10:48:302
26131997655,15cyclictest8380-21latency_hist10:57:342
26129997638,7cyclictest181rcu_preempt08:52:471
26129997631,16cyclictest0-21swapper/109:08:381
26129997622,24cyclictest0-21swapper/112:22:581
26122997661,11cyclictest26115-21cyclictest10:22:570
26122997644,23cyclictest24142-21grep12:17:530
2612299764,33cyclictest29697-21modprobe07:07:310
26122997622,33cyclictest0-21swapper/009:13:240
26131997556,13cyclictest20591-21fschecks_time08:17:552
26131997556,13cyclictest19384-21mailstats08:13:052
26131997556,13cyclictest19384-21mailstats08:13:052
26131997555,14cyclictest6039-21sh09:04:432
26131997555,14cyclictest6039-21sh09:04:432
26129997557,13cyclictest26115-21cyclictest07:28:071
26129997544,8cyclictest181rcu_preempt07:12:471
26129997542,10cyclictest181rcu_preempt11:26:001
26129997527,24cyclictest0-21swapper/107:21:561
26122997560,9cyclictest19304-21ssh12:09:220
2612299753,38cyclictest31844-21fschecks_time07:12:520
26129997445,15cyclictest181rcu_preempt07:54:411
26129997444,8cyclictest181rcu_preempt07:37:481
26129997444,7cyclictest181rcu_preempt08:10:581
26129997444,7cyclictest181rcu_preempt08:10:581
26129997442,10cyclictest181rcu_preempt10:37:211
26129997442,10cyclictest181rcu_preempt10:37:211
26129997424,18cyclictest0-21swapper/111:14:141
26122997452,16cyclictest17228-21timerandwakeup09:23:080
26122997452,16cyclictest17228-21timerandwakeup09:23:080
26131997355,12cyclictest2990-21mailstats07:23:052
26131997352,15cyclictest22773-21/usr/sbin/munin10:27:572
26131997352,14cyclictest18681-21latency_hist08:12:352
26131997352,14cyclictest18681-21latency_hist08:12:352
26131997351,14cyclictest15309-21latency_hist08:02:332
26131997335,23cyclictest20526-21apt-key12:12:332
26122997356,12cyclictest26115-21cyclictest10:57:010
2612299735,37cyclictest29031-21cut10:37:530
2612299733,31cyclictest352-21runrttasks09:00:120
26122997315,31cyclictest0-21swapper/011:39:540
26131997254,11cyclictest13665-21sh11:05:572
26131997254,11cyclictest13665-21sh11:05:572
26131997251,14cyclictest807-21diskmemload10:38:522
26129997213,21cyclictest301ktimers/107:02:351
2612299726,26cyclictest0-21swapper/010:04:040
2612299726,26cyclictest0-21swapper/010:04:040
26122997224,30cyclictest0-21swapper/012:26:340
26131997150,14cyclictest25125-21latency_hist08:32:342
26131997150,12cyclictest0-21swapper/210:12:472
26131997145,14cyclictest20131-21kworker/0:1-cgroup_destroy08:37:342
26129997148,8cyclictest181rcu_preempt07:48:041
2612299718,34cyclictest3724-21readlink07:27:310
26122997149,16cyclictest17584-21latency_hist12:07:340
2612299714,37cyclictest11791-21grep10:08:070
2612299713,41cyclictest13821-21latency09:17:570
2612299712,29cyclictest10023-21ssh11:53:220
26122997119,25cyclictest0-21swapper/010:59:210
26131997055,9cyclictest30851-21ssh11:34:052
26131997053,10cyclictest3860-21runrttasks09:02:022
26131997035,24cyclictest7145-21grep07:37:352
2613199702,56cyclictest28163-21apt-get08:42:452
26122997055,10cyclictest15350irq/295-2188000.ethernet10:47:470
26122997055,10cyclictest14706-21kworker/u17:0+rpciod09:44:380
26122997055,10cyclictest14706-21kworker/u17:0+rpciod09:44:380
26122997043,20cyclictest7068-21kworker/u17:3+nfsiod09:10:440
2612299703,34cyclictest26422-21/usr/sbin/munin11:27:400
2612299703,34cyclictest26422-21/usr/sbin/munin11:27:400
2612299703,29cyclictest27074-21ssh09:40:050
2612299703,29cyclictest27074-21ssh09:40:050
2613199698,31cyclictest277-21dbus-daemon10:27:332
26131996954,8cyclictest0-21swapper/209:08:022
26131996952,11cyclictest29150-21fschecks_count09:44:072
26131996952,11cyclictest29150-21fschecks_count09:44:072
26131996951,12cyclictest923-21runrttasks09:50:242
26131996951,12cyclictest0-21swapper/212:18:002
26131996950,14cyclictest18853-21sh09:27:052
26131996950,12cyclictest0-21swapper/209:58:092
26131996949,14cyclictest810-21cpu07:17:502
26131996948,15cyclictest11031-21sh09:13:052
26131996927,16cyclictest14046-21ps09:18:052
26122996950,13cyclictest352-21runrttasks09:28:050
2612299694,36cyclictest30241-21switchtime11:33:060
26122996943,19cyclictest2223-21fschecks_count09:52:530
26122996943,19cyclictest2223-21fschecks_count09:52:530
26122996936,23cyclictest32233-21nfs4_client10:43:000
26122996936,23cyclictest32233-21nfs4_client10:43:000
2612299693,35cyclictest29980-21/usr/sbin/munin07:07:530
26131996848,14cyclictest11241-21grep10:07:542
26131996847,14cyclictest16229-21switchtime08:03:042
26131996843,14cyclictest0-21swapper/212:02:332
26131996836,23cyclictest29696-21modprobe07:07:312
26122996841,20cyclictest0-21swapper/007:32:570
2612299683,33cyclictest12506-21fschecks_count07:52:530
2612299683,32cyclictest10576-21latency_hist07:47:580
2612299683,32cyclictest10576-21latency_hist07:47:580
2612299683,27cyclictest0-21swapper/011:20:220
2612299682,32cyclictest23994-21/usr/sbin/munin11:22:580
2612299682,29cyclictest14865-21fschecks_time11:07:530
26131996751,10cyclictest8836-21apt-get11:52:432
26131996750,11cyclictest26115-21cyclictest10:07:352
26131996750,11cyclictest26115-21cyclictest10:07:352
26131996725,32cyclictest31380-21grep07:12:332
26131996717,11cyclictest27178-21fschecks_time12:22:522
26131996710,30cyclictest39-21ksoftirqd/212:05:252
26122996751,9cyclictest4357-21kernelversion07:27:570
2612299674,44cyclictest1665-21unixbench_multi07:18:100
2612299674,29cyclictest18502-21grep08:12:310
2612299674,29cyclictest18502-21grep08:12:310
2612299673,54cyclictest23854-21ssh10:28:070
2612299673,27cyclictest18571-21ssh10:19:560
2612299673,27cyclictest0-21swapper/008:48:030
2612299673,27cyclictest0-21swapper/008:48:030
2612299673,27cyclictest0-21swapper/008:02:150
26136996611,31cyclictest0-21swapper/312:12:473
26131996651,9cyclictest17357-21latency_hist11:12:332
26131996647,12cyclictest0-21swapper/207:28:082
26131996610,29cyclictest39-21ksoftirqd/210:37:092
26131996610,29cyclictest39-21ksoftirqd/210:37:092
2612299664,45cyclictest352-21runrttasks10:33:120
2612299664,45cyclictest352-21runrttasks10:33:120
26131996546,13cyclictest17386-21netstat10:18:002
26131996530,25cyclictest23198-21latency_hist11:22:332
26131996516,16cyclictest24091-21ssh09:35:022
26131996514,14cyclictest6718-21sort11:48:072
26122996536,20cyclictest20090-21modprobe08:17:300
26122996536,20cyclictest20090-21modprobe08:17:300
2612299653,50cyclictest27907-21munin-node08:32:420
2612299653,24cyclictest0-21swapper/008:23:030
2613699645,40cyclictest0-21swapper/307:43:193
2613699645,40cyclictest0-21swapper/307:43:193
2613199643,40cyclictest26571-21fschecks_time06:57:542
2612299642,27cyclictest16721-21seq08:05:550
26136996316,32cyclictest0-21swapper/309:42:583
26136996316,32cyclictest0-21swapper/309:42:583
2613199633,39cyclictest19907-21ssh11:16:472
2613199633,24cyclictest405-21seq07:16:462
2613199632,36cyclictest25561-21fschecks_time09:37:532
2613199632,36cyclictest25561-21fschecks_time09:37:532
26131996322,8cyclictest181rcu_preempt07:48:202
2613199632,28cyclictest1434-21rm11:39:172
26131996314,17cyclictest31596-21chrt08:51:592
26131996314,17cyclictest31596-21chrt08:51:592
2612299633,25cyclictest0-21swapper/006:57:500
26131996250,7cyclictest0-21swapper/207:37:472
26131996239,16cyclictest0-21swapper/207:57:312
26131996238,15cyclictest1879-21latency_hist10:47:352
26131996238,15cyclictest1879-21latency_hist10:47:352
26131996215,11cyclictest0-21swapper/209:53:172
26131996215,11cyclictest0-21swapper/209:53:172
26131996146,8cyclictest0-21swapper/211:30:502
26131996146,8cyclictest0-21swapper/211:30:502
26131996133,20cyclictest22199-21sort08:22:532
26131996113,11cyclictest28024-21taskset08:40:542
2612299603,34cyclictest20350-21/usr/sbin/munin08:18:010
2613699572,45cyclictest9968-21taskset10:58:513
2613699563,40cyclictest3128-21chrt10:48:253
26136995541,9cyclictest451rcuc/311:37:013
2613699552,40cyclictest31822-21taskset07:12:523
26136995434,14cyclictest451rcuc/309:07:133
26136995434,14cyclictest451rcuc/309:07:133
2613699543,40cyclictest29758-21chrt10:38:073
26136995339,9cyclictest451rcuc/309:57:523
2613699533,38cyclictest2605-21chrt08:58:453
2613699533,36cyclictest2380-21chrt10:47:363
2613699533,34cyclictest5702-21chrt11:47:343
2613699533,34cyclictest30884-21taskset08:48:033
2613699533,34cyclictest30884-21taskset08:48:033
2613699532,37cyclictest13900-21chrt11:06:373
2613699532,37cyclictest13900-21chrt11:06:373
2613699523,36cyclictest10385-21chrt09:12:433
2613699523,35cyclictest27871-21chrt10:36:013
2613699523,35cyclictest27871-21chrt10:36:013
2613699522,36cyclictest7404-21chrt09:07:453
2613699522,36cyclictest32687-21taskset11:37:533
2613699522,36cyclictest24302-21chrt08:28:063
2613699522,35cyclictest4305-21taskset07:27:553
2613699522,35cyclictest4283-21chrt09:56:333
2613699522,35cyclictest4283-21chrt09:56:333
2613699513,35cyclictest14807-21taskset10:13:073
2613699513,34cyclictest24170-21chrt12:17:543
2613699513,32cyclictest22236-21chrt09:32:343
2613699512,38cyclictest8201-21taskset10:57:333
2613699512,36cyclictest22757-21chrt10:27:383
2613699512,36cyclictest18875-21chrt10:20:493
2613699512,35cyclictest10807-21chrt11:55:363
2613699512,34cyclictest32104-21chrt09:48:023
2613699512,34cyclictest1204-21taskset08:57:353
2613699505,35cyclictest23056-21chrt08:24:213
26136995034,9cyclictest451rcuc/308:16:113
26136995034,9cyclictest451rcuc/308:16:113
2613699503,37cyclictest24409-21chrt11:23:063
2613699503,35cyclictest27225-21chrt11:28:053
2613699503,35cyclictest27225-21chrt11:28:053
2613699502,36cyclictest28736-21taskset08:43:293
2613699502,35cyclictest25545-21chrt09:37:533
2613699502,35cyclictest25545-21chrt09:37:533
2613699498,27cyclictest19613-21chrt12:10:153
26136994936,9cyclictest451rcuc/307:06:213
26136994935,9cyclictest451rcuc/308:32:373
26136994934,8cyclictest451rcuc/311:16:343
26136994933,9cyclictest451rcuc/309:21:273
2613699493,33cyclictest8571-21chrt11:52:323
2613699493,32cyclictest790-21chrt10:44:283
2613699493,32cyclictest15404-21chrt08:02:343
2613699492,35cyclictest15035-21chrt12:02:543
2613699492,34cyclictest27187-21chrt12:22:523
2613699492,34cyclictest12565-21taskset10:09:433
2613699492,33cyclictest24090-21taskset09:35:013
247302499,24sleep30-21swapper/306:53:303
2613699482,37cyclictest10394-21taskset10:07:033
2613699482,37cyclictest10394-21taskset10:07:033
2613699482,31cyclictest7326-21chrt07:37:473
2613699473,33cyclictest14612-21chrt12:02:333
2613699473,30cyclictest21406-21taskset10:24:343
2613699472,33cyclictest13667-21chrt07:57:323
2613699472,33cyclictest10794-21taskset07:48:043
2613699472,32cyclictest3283-21chrt07:23:093
2613699472,32cyclictest16810-21chrt09:22:573
2613699472,32cyclictest16810-21chrt09:22:573
2613699472,31cyclictest15874-21chrt08:02:543
2613699453,32cyclictest14534-21taskset11:07:383
26136994431,9cyclictest451rcuc/307:09:493
26136994431,8cyclictest451rcuc/307:32:503
26136994430,9cyclictest451rcuc/308:42:133
26136994422,15cyclictest451rcuc/306:58:023
2613699432,31cyclictest21737-21taskset11:18:323
26136994228,10cyclictest451rcuc/308:18:233
2613699403,28cyclictest18234-21taskset08:09:193
2613699403,28cyclictest18234-21taskset08:09:193
2613699392,27cyclictest1984-21taskset07:21:183
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional