You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-11 - 11:58
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Thu Dec 11, 2025 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2656625850,8sleep30-21swapper/319:08:123
2509121539,25sleep00-21swapper/019:03:290
26548991233,81cyclictest1497-21apt-get21:08:450
26548991182,27cyclictest27584-21/usr/sbin/munin23:43:400
26548991179,75cyclictest0-21swapper/022:38:110
265489911436,64cyclictest0-21swapper/000:23:200
26548991142,25cyclictest11566-21ssh00:13:060
25286211311,71sleep225292-21/usr/sbin/munin19:04:042
26548991123,28cyclictest10763-21latency_hist19:58:110
26548991112,25cyclictest4687-21ssh23:59:490
26548991112,24cyclictest20875-21rm21:41:010
26548991103,68cyclictest22120-21grep23:33:450
26548991103,68cyclictest22120-21grep23:33:450
26548991092,27cyclictest21505-21ssh00:29:340
26548991092,27cyclictest21505-21ssh00:29:340
26548991087,66cyclictest0-21swapper/023:55:510
26548991082,64cyclictest6226-21tune2fs19:43:320
26548991082,27cyclictest13391-21cat22:23:430
26548991082,27cyclictest13391-21cat22:23:430
26548991082,24cyclictest15914-21runrttasks22:28:580
265639910746,10cyclictest0-21swapper/200:08:192
26548991072,27cyclictest32099-21ssh22:56:270
26548991072,27cyclictest32099-21ssh22:56:270
26548991072,23cyclictest13304-21ssh00:14:470
26548991072,23cyclictest13304-21ssh00:14:470
265489910717,57cyclictest0-21swapper/023:38:370
265639910652,8cyclictest0-21swapper/200:03:102
265489910627,52cyclictest0-21swapper/023:48:450
265489910627,52cyclictest0-21swapper/023:48:450
26548991062,22cyclictest10519-21rm23:14:230
26548991052,88cyclictest22964-21sshd00:33:250
26548991052,26cyclictest17450-21cstates20:18:250
26548991052,26cyclictest17450-21cstates20:18:250
26548991052,24cyclictest394-21ssh22:02:190
26548991052,24cyclictest15957-21ssh00:19:130
265489910517,52cyclictest0-21swapper/022:03:380
265639910349,12cyclictest22924-21sh00:33:202
265489910368,24cyclictest25354-21latency_hist20:43:110
26548991032,88cyclictest4296-21ssh21:13:240
26548991032,67cyclictest18760-21grep23:28:360
26548991032,27cyclictest5494-21ssh22:10:160
26548991032,27cyclictest5494-21ssh22:10:160
26548991032,25cyclictest25760-21ssh22:44:340
265489910231,62cyclictest0-21swapper/021:08:120
26548991022,28cyclictest27657-21/usr/sbin/munin21:53:400
26548991022,28cyclictest27657-21/usr/sbin/munin21:53:400
265639910143,10cyclictest0-21swapper/221:13:312
26548991014,7cyclictest0-21swapper/021:33:240
26548991012,6cyclictest0-21swapper/022:38:220
26548991012,6cyclictest0-21swapper/022:38:220
26548991012,69cyclictest12934-21/usr/sbin/munin21:28:090
26548991012,27cyclictest2112-21ssh23:00:090
26548991003,83cyclictest11813-21apt-get23:18:170
26563999978,15cyclictest623-21cpuspeed22:58:252
2654899992,21cyclictest29439-21timerandwakeup20:53:450
26563999836,10cyclictest0-21swapper/220:58:122
26548999841,50cyclictest20849-21cut20:28:310
2654899983,85cyclictest6183-21ssh00:03:250
2654899983,85cyclictest6183-21ssh00:03:250
26548999835,54cyclictest0-21swapper/022:13:390
26563999744,10cyclictest0-21swapper/221:23:452
2656399972,42cyclictest11405-21grep19:58:392
26548999740,48cyclictest0-21swapper/019:33:390
2654899973,7cyclictest0-21swapper/019:08:200
26548999733,58cyclictest0-21swapper/019:13:320
2654899972,26cyclictest5427-21chrt19:38:580
2654899972,22cyclictest26294-21ssh21:49:580
2654899972,21cyclictest9909-21ssh21:22:480
2654899962,7cyclictest0-21swapper/020:43:200
2656399952,45cyclictest7080-21fschecks_time22:13:302
26555999545,12cyclictest10251-21ntp_kernel_err22:18:361
2654899953,63cyclictest7623-21ssh23:09:220
2654899953,23cyclictest20506-21ls20:28:110
2654899953,23cyclictest20506-21ls20:28:110
2654899952,7cyclictest0-21swapper/022:48:200
2654899944,82cyclictest0-21swapper/022:18:240
2654899942,25cyclictest22179-21apt-get20:33:200
2654899942,23cyclictest4751-21ssh23:04:240
2654899932,5cyclictest0-21swapper/021:43:220
2654899932,5cyclictest0-21swapper/021:43:220
26555999273,8cyclictest0-21swapper/120:28:431
2654899922,82cyclictest10866-21/usr/sbin/munin19:58:490
2654899922,26cyclictest15222-21ssh21:31:300
2654899922,26cyclictest15222-21ssh21:31:300
2654899913,81cyclictest0-21swapper/019:23:250
2654899912,30cyclictest8282-21grep19:48:390
2654899912,26cyclictest17212-21sort20:18:100
2654899912,26cyclictest17212-21sort20:18:100
2654899912,26cyclictest12902-21latency20:03:340
2654899903,78cyclictest792-21diskmemload23:23:360
2654899903,78cyclictest30400-21/usr/sbin/munin19:18:440
2654899902,81cyclictest27660-21sendmail_mailst20:48:420
2654899902,6cyclictest0-21swapper/020:58:250
2656399883,39cyclictest24366-21/usr/sbin/munin22:43:362
2654899883,5cyclictest0-21swapper/020:08:210
26563998758,8cyclictest181rcu_preempt21:33:382
2654899872,24cyclictest1534-21uname19:28:340
26563998555,21cyclictest5405-21sh23:06:212
26563998260,16cyclictest26537-21cyclictest19:33:252
26563998255,8cyclictest181rcu_preempt22:23:282
26563998255,8cyclictest181rcu_preempt22:23:282
2657199817,54cyclictest0-21swapper/321:18:303
26563998161,13cyclictest1-21systemd00:21:252
26563998153,21cyclictest20202-21kworker/u19:0+nfsiod21:58:262
26555998165,10cyclictest9583-21if_eth019:53:331
26563998065,9cyclictest5889-21kworker/u17:1-xprtiod23:30:092
26563998060,9cyclictest0-21swapper/222:03:372
26555998041,8cyclictest181rcu_preempt00:03:261
26555998041,8cyclictest181rcu_preempt00:03:261
26563997933,13cyclictest0-21swapper/222:33:432
26563997846,12cyclictest181rcu_preempt21:53:232
26563997846,12cyclictest181rcu_preempt21:53:232
26555997862,9cyclictest0-21swapper/121:23:421
26555997855,17cyclictest27317-21kernelversion23:43:321
26555997853,9cyclictest31-21ksoftirqd/121:42:031
26563997737,16cyclictest0-21swapper/200:28:342
26563997737,16cyclictest0-21swapper/200:28:342
26555997757,11cyclictest21419-21ssh22:38:111
26555997756,14cyclictest26537-21cyclictest23:18:411
26555997742,18cyclictest5907-21rm21:15:181
26555997725,8cyclictest181rcu_preempt21:53:241
26555997725,8cyclictest181rcu_preempt21:53:241
26563997661,8cyclictest0-21swapper/220:23:362
26563997661,8cyclictest0-21swapper/220:23:362
26555997638,9cyclictest181rcu_preempt00:08:231
26555997632,9cyclictest181rcu_preempt23:38:101
26555997632,9cyclictest181rcu_preempt23:38:101
26563997560,8cyclictest0-21swapper/223:08:442
26563997552,8cyclictest181rcu_preempt23:43:412
26555997535,21cyclictest24060-21modprobe22:43:081
26555997535,21cyclictest24060-21modprobe22:43:081
26563997459,9cyclictest176-21jbd2/mmcblk2p2-822:22:432
26555997455,8cyclictest0-21swapper/123:08:411
26563997358,8cyclictest0-21swapper/220:08:402
26563997355,13cyclictest15548-21cut23:23:312
26555997357,11cyclictest30090-21latency_hist20:58:101
26555997326,13cyclictest181rcu_preempt22:13:531
26563997257,9cyclictest7129-21timerwakeupswit00:03:472
26563997257,9cyclictest7129-21timerwakeupswit00:03:472
26563997253,10cyclictest0-21swapper/220:03:302
26563997250,8cyclictest181rcu_preempt19:58:102
26555997255,11cyclictest369-21ntpd23:26:551
26555997254,12cyclictest355-21runrttasks23:03:361
26555997252,11cyclictest871-21sendmail-mta20:13:421
26555997252,11cyclictest871-21sendmail-mta20:13:421
2655599723,34cyclictest0-21swapper/121:48:281
26563997151,7cyclictest181rcu_preempt22:51:092
26563997148,17cyclictest16529-21sh22:29:252
26563997143,20cyclictest7992-21fschecks_time21:19:092
2655599717,23cyclictest11969-21apt-config23:18:101
26555997151,14cyclictest30017-21sh23:48:291
26555997151,14cyclictest30017-21sh23:48:291
2655599713,53cyclictest24364-21sh22:43:191
26563997054,9cyclictest0-21swapper/220:23:072
26563997054,9cyclictest0-21swapper/220:23:072
26563997053,8cyclictest0-21swapper/219:23:102
26555997055,8cyclictest0-21swapper/100:23:371
26555997052,12cyclictest989-21/usr/sbin/munin22:03:271
26555997052,12cyclictest2937-21systemd-run23:58:081
26555997050,14cyclictest691-21cron21:48:081
26555997050,14cyclictest691-21cron21:48:081
2508727010,36sleep124576-21apt-get19:03:261
26563996955,8cyclictest0-21swapper/219:08:432
26563996954,8cyclictest0-21swapper/219:48:112
26563996952,10cyclictest435-21journal-offline23:53:252
26555996934,9cyclictest0-21swapper/120:48:201
26563996854,10cyclictest26537-21cyclictest23:19:282
26563996851,11cyclictest25552-21awk21:48:442
26563996850,11cyclictest9198-21/usr/sbin/munin23:13:462
26563996847,7cyclictest181rcu_preempt22:10:402
26563996847,7cyclictest181rcu_preempt22:10:402
26563996844,7cyclictest181rcu_preempt22:40:302
26563996844,7cyclictest181rcu_preempt22:40:302
26563996841,8cyclictest181rcu_preempt19:13:272
26555996834,8cyclictest181rcu_preempt21:58:311
2655599683,45cyclictest19788-21timerwakeupswit20:23:461
2655599683,45cyclictest19788-21timerwakeupswit20:23:461
26555996833,11cyclictest683-21kworker/u17:3-rpciod22:23:251
26555996833,11cyclictest683-21kworker/u17:3-rpciod22:23:251
26563996745,12cyclictest0-21swapper/220:43:302
26563996741,7cyclictest181rcu_preempt23:48:452
26563996741,7cyclictest181rcu_preempt23:48:452
26563996741,10cyclictest181rcu_preempt19:48:392
26563996737,21cyclictest1607-21cstates21:08:512
26563996734,16cyclictest11803-21kworker/u17:0-events_unbound21:41:002
26555996752,9cyclictest0-21swapper/120:33:301
26555996752,8cyclictest0-21swapper/120:18:431
26555996752,8cyclictest0-21swapper/120:18:431
26555996752,8cyclictest0-21swapper/120:08:311
26555996747,14cyclictest24359-21idleruntime23:38:331
26555996747,11cyclictest0-21swapper/119:48:241
26555996746,14cyclictest9018-21sh21:20:071
26555996744,13cyclictest0-21swapper/100:18:251
26563996652,10cyclictest26537-21cyclictest21:48:002
26563996652,10cyclictest26537-21cyclictest21:48:002
26563996636,7cyclictest181rcu_preempt21:28:532
26563996636,7cyclictest181rcu_preempt21:28:532
26563996629,13cyclictest0-21swapper/220:43:102
26555996650,9cyclictest0-21swapper/121:33:441
26555996646,14cyclictest20554-21sh00:28:331
26555996646,14cyclictest20554-21sh00:28:331
26555996642,8cyclictest181rcu_preempt19:38:421
26555996637,20cyclictest26764-21modprobe20:48:081
2655599663,53cyclictest15692-21apt-get22:28:231
26555996631,10cyclictest19026-21ssh23:28:431
26563996548,8cyclictest0-21swapper/223:38:372
26563996536,20cyclictest5004-21wc19:38:422
26563996521,35cyclictest15909-21cut20:13:312
26563996521,35cyclictest15909-21cut20:13:312
26555996546,13cyclictest1-21systemd20:42:001
26555996545,14cyclictest32760-21latency_hist22:58:101
26555996545,14cyclictest32760-21latency_hist22:58:101
2655599653,22cyclictest13974-21/usr/sbin/munin21:28:441
2655599653,22cyclictest13974-21/usr/sbin/munin21:28:441
26555996510,45cyclictest1341-21apt-get21:08:241
2657199646,31cyclictest0-21swapper/322:29:013
2656399648,10cyclictest181rcu_preempt23:33:232
2656399648,10cyclictest181rcu_preempt23:33:232
26563996444,9cyclictest0-21swapper/221:08:102
26563996441,13cyclictest0-21swapper/200:13:262
26563996441,13cyclictest0-21swapper/200:13:262
26555996447,11cyclictest27890-21ssh22:48:431
26555996429,12cyclictest12319-21kworker/u18:2-rpciod22:58:421
26555996424,18cyclictest0-21swapper/122:09:481
26555996424,18cyclictest0-21swapper/122:09:481
2657199633,43cyclictest0-21swapper/323:18:223
2656399633,46cyclictest23609-21latency_hist20:38:102
2656399633,45cyclictest30639-21kernelversion20:58:342
26563996334,20cyclictest32601-21wc19:23:412
26555996347,10cyclictest369-21ntpd20:58:261
2655599634,42cyclictest376-21sshd00:17:251
2655599634,42cyclictest376-21sshd00:17:251
2655599633,40cyclictest355-21runrttasks19:29:451
26555996328,8cyclictest0-21swapper/119:08:441
2655599632,36cyclictest29261-21ntp_kernel_err19:13:381
2655599632,31cyclictest6878-21timerwakeupswit19:43:461
2655599624,41cyclictest2886-21cstates19:33:251
26555996239,17cyclictest23783-21threads00:33:451
2655599622,35cyclictest4076-21find23:58:431
26555996216,22cyclictest0-21swapper/120:03:311
26563996148,8cyclictest0-21swapper/222:53:272
26563996148,8cyclictest0-21swapper/222:53:272
26563996140,8cyclictest181rcu_preempt00:23:262
26563996127,11cyclictest0-21swapper/219:29:552
26563996040,9cyclictest0-21swapper/220:48:322
2656399603,45cyclictest355-21runrttasks20:31:522
2655599603,43cyclictest30669-21cut19:18:321
2655599602,39cyclictest11832-21unixbench_singl19:58:461
26571995911,27cyclictest0-21swapper/321:08:303
2655599585,25cyclictest32198-21fschecks_count19:23:311
2655599583,37cyclictest32546-21mailstats21:03:401
2657199574,37cyclictest18914-21taskset21:38:113
2657199573,37cyclictest31748-21chrt21:03:103
2657199564,37cyclictest10616-21taskset19:58:083
2657199563,40cyclictest16418-21chrt20:13:433
2657199563,40cyclictest16418-21chrt20:13:433
2657199554,40cyclictest15596-21taskset22:28:123
2657199554,40cyclictest15596-21taskset22:28:123
2657199553,43cyclictest18774-21taskset23:28:373
2657199553,39cyclictest9599-21chrt00:08:373
2657199553,38cyclictest27050-21taskset23:43:243
2657199552,40cyclictest23477-21taskset22:41:253
2657199552,40cyclictest23477-21taskset22:41:253
2657199544,38cyclictest4872-21taskset22:08:473
2657199544,38cyclictest4872-21taskset22:08:473
2657199543,37cyclictest21039-21chrt23:33:153
2657199543,37cyclictest21039-21chrt23:33:153
2657199537,40cyclictest26788-21chrt20:48:083
2657199533,41cyclictest11670-21taskset23:17:473
2657199533,39cyclictest6997-21chrt21:18:093
2657199533,39cyclictest17539-21chrt20:18:293
2657199533,39cyclictest17539-21chrt20:18:293
2657199533,36cyclictest3809-21chrt23:03:333
2657199533,35cyclictest23903-21taskset20:38:243
2657199525,36cyclictest2902-21chrt23:58:073
2657199524,38cyclictest6498-21taskset00:03:343
2657199524,38cyclictest6498-21taskset00:03:343
2657199523,40cyclictest18761-21chrt22:33:273
2657199523,38cyclictest4173-21chrt19:38:083
2657199523,38cyclictest13423-21chrt21:28:313
2657199523,38cyclictest13423-21chrt21:28:313
2657199523,35cyclictest7611-21taskset23:09:213
2657199523,35cyclictest13648-21chrt20:05:543
2657199522,37cyclictest876-21taskset22:58:313
2657199514,34cyclictest25647-21taskset21:48:453
2657199514,33cyclictest29606-21chrt20:53:473
2657199514,33cyclictest11570-21taskset21:24:093
2657199513,38cyclictest22900-21chrt21:43:423
2657199513,38cyclictest22900-21chrt21:43:423
2657199513,38cyclictest2139-21chrt19:28:453
2657199513,38cyclictest19568-21chrt20:23:433
2657199513,38cyclictest19568-21chrt20:23:433
2657199513,35cyclictest17575-21taskset00:23:283
2657199512,38cyclictest969-21chrt22:03:153
2657199503,35cyclictest28438-21chrt20:53:103
2657199503,35cyclictest15481-21chrt20:13:103
2657199503,34cyclictest32752-21chrt22:58:103
2657199503,34cyclictest32752-21chrt22:58:103
2657199503,34cyclictest27963-21chrt22:48:463
2657199503,33cyclictest30919-21chrt23:48:473
2657199503,33cyclictest30919-21chrt23:48:473
2657199502,37cyclictest25924-21chrt22:45:053
2657199502,36cyclictest20426-21taskset00:28:303
2657199502,36cyclictest20426-21taskset00:28:303
26571994935,8cyclictest451rcuc/319:12:583
2657199493,37cyclictest28829-21chrt21:54:003
2657199493,37cyclictest28829-21chrt21:54:003
2657199493,36cyclictest4963-21taskset19:38:413
2657199493,35cyclictest22008-21chrt20:33:103
2657199493,33cyclictest19010-21chrt21:38:193
2657199493,33cyclictest12192-21chrt00:13:313
2657199493,33cyclictest12192-21chrt00:13:313
2657199493,31cyclictest16461-21chrt00:20:523
2657199492,35cyclictest8444-21chrt19:48:413
2657199492,34cyclictest32080-21chrt21:59:593
2657199492,33cyclictest10585-21chrt22:18:433
2657199483,35cyclictest17025-21chrt23:25:423
2657199483,33cyclictest1233-21chrt21:08:123
2657199483,32cyclictest6902-21chrt22:13:243
2657199482,33cyclictest6538-21chrt19:43:393
2657199482,32cyclictest24533-21chrt00:35:233
2657199474,34cyclictest30151-21chrt19:18:083
2657199473,35cyclictest4595-21taskset23:59:333
2657199462,31cyclictest30711-21chrt19:18:333
2657199453,30cyclictest23565-21chrt20:38:083
2657199452,29cyclictest25434-21chrt23:39:443
2657199443,29cyclictest762-21chrt19:26:223
2657199373,25cyclictest11031-21chrt19:58:293
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional