You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-25 - 07:11
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Thu Jun 25, 2026 00:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
89359912139,32cyclictest0-21swapper/023:03:370
89469911560,49cyclictest15847-21diskmemload21:20:371
89469911243,17cyclictest0-21swapper/120:58:251
89359910740,29cyclictest6543-21seq00:18:490
89359910435,26cyclictest0-21swapper/021:54:280
89359910354,39cyclictest19676-21snmpd23:13:320
89359910342,23cyclictest9220-21/usr/sbin/munin00:23:430
89469910274,22cyclictest20587-21ssh22:53:491
89359910233,30cyclictest24035-21ssh21:11:050
89359910140,22cyclictest4902-21ssh21:33:430
89359910041,23cyclictest28180-21ssh21:18:370
89359910037,27cyclictest4504-21ssh00:14:580
89359910036,25cyclictest21271-21ssh21:06:480
89359910036,25cyclictest21271-21ssh21:06:480
89359910031,25cyclictest0-21swapper/021:23:510
2979021000,7sleep20-21swapper/219:58:592
2979021000,7sleep20-21swapper/219:58:592
8935999940,22cyclictest11302-21seq19:03:440
8935999937,26cyclictest22692-21ssh22:58:090
8935999930,19cyclictest0-21swapper/019:38:590
8935999837,29cyclictest1695-21/usr/sbin/munin21:28:530
8935999744,41cyclictest20827-21/usr/sbin/munin19:34:020
8935999744,18cyclictest23303-21/usr/sbin/munin22:58:430
8935999744,18cyclictest23303-21/usr/sbin/munin22:58:430
8935999738,26cyclictest14175-21sh22:43:340
8935999737,24cyclictest25218-21sh22:07:360
8935999736,27cyclictest0-21swapper/021:43:380
8935999735,23cyclictest4773-21fschecks_time20:23:440
8935999640,23cyclictest29214-21ssh22:13:480
8935999639,21cyclictest29234-21ssh23:08:400
8935999638,26cyclictest10417-21ssh21:43:240
8935999638,26cyclictest10417-21ssh21:43:240
8935999637,24cyclictest11507-21sh23:33:380
8935999637,24cyclictest11507-21sh23:33:380
8935999635,26cyclictest2202-21rm00:11:510
8935999538,23cyclictest21507-21ssh23:49:010
8935999537,26cyclictest28163-21ssh22:12:290
8935999537,26cyclictest28163-21ssh22:12:290
8935999536,19cyclictest9301-21head22:33:590
8935999443,40cyclictest9163-21/usr/sbin/munin18:58:390
8935999438,23cyclictest17597-21taskset23:43:410
8935999437,22cyclictest9964-21proc_pri20:38:550
8935999435,21cyclictest0-21swapper/019:28:560
8935999433,23cyclictest3416-21taskset22:24:050
8946999372,14cyclictest29119-21fschecks_time22:13:461
8946999310,72cyclictest26565-21apt-get23:58:531
8935999340,19cyclictest23384-21ssh23:53:320
8935999339,20cyclictest6128-21sh22:28:570
8935999337,21cyclictest15150-21rm21:49:560
8935999337,21cyclictest15150-21rm21:49:560
8935999337,19cyclictest529-21timerwakeupswit20:08:590
8935999337,19cyclictest529-21timerwakeupswit20:08:590
8935999337,19cyclictest19676-21snmpd23:59:390
8935999335,24cyclictest17314-21ssh20:59:070
8935999335,24cyclictest17314-21ssh20:59:070
8935999334,26cyclictest0-21swapper/023:18:350
8935999334,26cyclictest0-21swapper/023:18:350
8935999333,25cyclictest8474-21/usr/sbin/munin23:28:440
8935999330,23cyclictest0-21swapper/023:41:460
8935999240,22cyclictest352-21runrttasks20:48:360
8935999238,23cyclictest15994-21sed20:58:250
8935999238,21cyclictest15847-21diskmemload00:03:370
8935999237,24cyclictest21255-21rm21:59:460
8935999237,24cyclictest21255-21rm21:59:460
8935999236,22cyclictest18620-21seq19:24:180
8935999139,18cyclictest11246-21ssh22:38:390
8935999137,21cyclictest19724-21sh22:53:070
8935999136,23cyclictest6965-21rm23:25:020
8935999130,22cyclictest0-21swapper/021:15:140
8935999130,22cyclictest0-21swapper/021:15:140
8935999048,32cyclictest0-21swapper/019:08:340
8935999036,23cyclictest1232-21rm22:21:200
8935999034,21cyclictest0-21swapper/020:28:540
8935998955,24cyclictest8921-21cyclictest18:58:270
8935998955,24cyclictest8921-21cyclictest18:58:270
8935998936,22cyclictest27291-21/usr/sbin/munin19:53:480
8935998936,22cyclictest27291-21/usr/sbin/munin19:53:480
779228911,39sleep20-21swapper/218:53:522
779228911,39sleep20-21swapper/218:53:522
8935998859,20cyclictest32061-21latency_hist20:08:260
8935998840,18cyclictest2654-21modprobe20:18:220
8935998840,18cyclictest2654-21modprobe20:18:220
8935998839,18cyclictest29897-21seq19:59:010
8935998839,18cyclictest29897-21seq19:59:010
8935998833,21cyclictest0-21swapper/020:33:490
8935998830,20cyclictest0-21swapper/019:13:410
8946998751,13cyclictest181rcu_preempt23:53:351
8946998657,21cyclictest14067-21latency_hist22:43:251
8935998636,20cyclictest16152-21df19:18:420
8935998636,20cyclictest16152-21df19:18:420
8935998633,19cyclictest0-21swapper/020:43:470
8955998562,15cyclictest11826-21sh00:28:242
8955998557,21cyclictest10423-21kworker/u17:3-rpciod22:28:582
8955998556,22cyclictest22410-21latency_hist19:38:262
894699851,42cyclictest12536-21tr19:08:231
8955998458,18cyclictest15847-21diskmemload21:10:252
8955998362,14cyclictest14015-21latency_hist22:43:252
8955998357,18cyclictest23075-21sh22:58:322
8955998357,18cyclictest23075-21sh22:58:322
8935998333,22cyclictest0-21swapper/019:48:520
8955998260,16cyclictest19054-21latency_hist19:28:252
8955998234,33cyclictest31379-21fschecks_count21:23:452
8946998260,15cyclictest0-21swapper/121:58:521
8946998260,15cyclictest0-21swapper/121:58:521
8935998244,29cyclictest24042-21/usr/sbin/munin19:43:330
877528213,44sleep18809-21munin-run18:58:231
877528213,44sleep18809-21munin-run18:58:231
8946998152,9cyclictest181rcu_preempt19:38:381
8946998134,14cyclictest181rcu_preempt20:13:221
8946998134,14cyclictest181rcu_preempt20:13:221
8935998152,20cyclictest2928-21apt-get20:18:390
8935998152,20cyclictest2928-21apt-get20:18:390
312752810,21sleep1301ktimers/123:12:111
8955998059,15cyclictest15847-21diskmemload23:41:032
8946998057,14cyclictest14647-21kworker/u18:1+flush-179:022:28:471
8946998055,15cyclictest2517-21diskmemload22:23:431
8946998052,22cyclictest14647-21kworker/u18:1+nfsiod23:46:591
8955997955,15cyclictest15847-21diskmemload21:35:122
8955997951,19cyclictest17558-21latency_hist19:23:262
8955997951,19cyclictest17558-21latency_hist19:23:262
8946997959,14cyclictest21867-21munin-plugin-st21:08:221
8946997959,14cyclictest21867-21munin-plugin-st21:08:221
894699793,47cyclictest8144-21grep21:38:481
894699793,47cyclictest8144-21grep21:38:481
8955997857,15cyclictest8921-21cyclictest21:51:252
8955997857,15cyclictest8921-21cyclictest21:51:252
8955997854,17cyclictest22787-21sh22:03:272
8946997862,10cyclictest15847-21diskmemload21:53:261
8946997862,10cyclictest15847-21diskmemload21:53:261
894699784,62cyclictest3088-21ssh00:13:361
896199774,38cyclictest0-21swapper/323:58:403
8955997758,12cyclictest0-21swapper/219:58:232
8955997758,12cyclictest0-21swapper/219:58:232
8955997754,16cyclictest25886-21cut19:48:442
8955997744,18cyclictest1970-21idleruntime-cro22:23:222
895599773,62cyclictest12870-21apt-get20:48:372
894699774,56cyclictest8167-21munin-run23:28:221
8946997720,14cyclictest181rcu_preempt19:53:581
8946997720,14cyclictest181rcu_preempt19:53:581
226452770,8sleep20-21swapper/223:52:122
8955997651,17cyclictest9603-21ntp_kernel_pll_18:58:502
8955997647,19cyclictest12722-21latency_hist19:08:262
8946997646,9cyclictest181rcu_preempt21:01:561
8946997646,9cyclictest181rcu_preempt21:01:561
894699763,52cyclictest12865-21sshd21:47:011
894699763,28cyclictest10535-21grep19:03:251
8955997526,20cyclictest0-21swapper/221:04:012
8955997526,20cyclictest0-21swapper/221:04:012
8946997559,10cyclictest15030-21ntp_kernel_pll_23:38:511
8946997556,13cyclictest25402-21latency_hist19:48:261
8946997553,15cyclictest2374-21systemd-journal23:51:241
894699749,53cyclictest22436-21fschecks_count21:08:431
8946997456,12cyclictest26655-21ntp_kernel_pll_23:03:511
8946997454,14cyclictest2779-21latency_hist20:18:251
8946997454,14cyclictest2779-21latency_hist20:18:251
8946997452,16cyclictest5937-21sh21:34:221
742827410,38sleep30-21swapper/318:53:323
742827410,38sleep30-21swapper/318:53:323
8955997355,13cyclictest26252-21sh22:08:492
8955997355,13cyclictest26252-21sh22:08:492
8955997351,15cyclictest0-21swapper/200:13:442
8955997349,16cyclictest8921-21cyclictest22:35:052
8955997316,17cyclictest25843-21proc_pri21:13:542
8955997316,17cyclictest25843-21proc_pri21:13:542
894699736,47cyclictest12870-21apt-get20:48:451
8946997349,16cyclictest0-21swapper/122:21:111
8946997346,19cyclictest17520-21fschecks_time21:54:381
8946997338,12cyclictest181rcu_preempt22:33:451
894699733,48cyclictest4404-21modprobe21:33:221
8946997319,21cyclictest3021-21cstates20:18:401
8946997319,21cyclictest3021-21cstates20:18:401
8955997252,14cyclictest15847-21diskmemload21:54:312
895599723,36cyclictest26245-21apt-key23:58:252
8955997215,10cyclictest30838-21munin-plugin-st21:23:222
8955997212,10cyclictest181rcu_preempt00:13:222
8955997153,13cyclictest8921-21cyclictest20:43:552
8955997146,16cyclictest352-21runrttasks21:31:422
8955997144,16cyclictest13532-21sh21:48:262
894699716,32cyclictest23883-21grep22:04:011
894699713,46cyclictest11581-21ps20:43:531
894699713,26cyclictest13740-21head19:08:581
8946997123,41cyclictest301ktimers/122:43:331
894699712,22cyclictest25174-21apt-get21:13:331
894699712,22cyclictest25174-21apt-get21:13:331
8955997051,11cyclictest29546-21apt-get00:03:332
8955997050,15cyclictest17021-21kworker/u18:3+events_unbound22:18:222
8955997048,11cyclictest0-21swapper/223:08:252
8955997046,16cyclictest352-21runrttasks21:38:402
8955997046,16cyclictest352-21runrttasks21:38:402
895599704,27cyclictest3151-21fschecks_time20:18:452
895599704,27cyclictest3151-21fschecks_time20:18:452
895599702,31cyclictest32612-21ntpq23:13:502
8946997046,13cyclictest0-21swapper/123:33:241
894699704,45cyclictest13226-21ssh23:35:521
894699704,45cyclictest13226-21ssh23:35:521
894699703,27cyclictest9809-21grep00:23:571
8955996955,10cyclictest8921-21cyclictest23:29:112
8955996928,11cyclictest181rcu_preempt20:33:252
8955996917,27cyclictest15847-21diskmemload22:49:002
8946996951,12cyclictest8921-21cyclictest20:38:501
894699694,13cyclictest0-21swapper/122:48:351
894699693,46cyclictest17545-21latency_hist19:23:261
894699693,46cyclictest17545-21latency_hist19:23:261
894699693,40cyclictest29797-21fschecks_count00:03:441
8955996843,12cyclictest0-21swapper/219:33:262
895599684,29cyclictest8162-21latency_hist23:28:262
895599683,30cyclictest20464-21fschecks_time22:53:462
895599682,28cyclictest15790-21ssh22:45:282
8955996816,12cyclictest17394-21apt-get23:43:362
8946996851,12cyclictest58-21kcompactd020:28:481
8946996840,13cyclictest0-21swapper/122:08:401
8946996840,13cyclictest0-21swapper/122:08:401
894699683,43cyclictest2219-21latency_hist23:18:251
894699683,43cyclictest20658-21latency_hist19:33:251
8961996739,19cyclictest30298-21chrt00:03:563
895599676,26cyclictest32606-21seq20:08:492
895599676,26cyclictest32606-21seq20:08:492
8955996751,9cyclictest0-21swapper/219:43:442
8955996737,20cyclictest31881-21munin-plugin-st20:08:252
8946996749,11cyclictest23473-21if_err_eth022:58:461
8946996749,11cyclictest23473-21if_err_eth022:58:461
8946996746,15cyclictest19676-21snmpd21:27:041
79472670,7sleep10-21swapper/120:33:451
895599661,26cyclictest14401-21seq19:13:362
894699664,39cyclictest18952-21modprobe19:28:221
8946996643,12cyclictest0-21swapper/123:18:381
8946996643,12cyclictest0-21swapper/123:18:381
894699663,43cyclictest26620-21grep19:49:001
8946996612,14cyclictest181rcu_preempt00:18:441
8955996546,12cyclictest297-21systemd-logind19:13:242
8955996539,13cyclictest0-21swapper/220:18:242
8955996539,13cyclictest0-21swapper/220:18:242
895599653,38cyclictest15986-21latency_hist20:58:252
8955996528,11cyclictest181rcu_preempt23:18:562
8955996528,11cyclictest181rcu_preempt23:18:562
894699654,19cyclictest21123-21fschecks_time19:33:451
894699653,35cyclictest31024-21find20:03:571
894699653,35cyclictest31024-21find20:03:571
8946996522,33cyclictest5231-21cat20:23:561
8955996447,12cyclictest8921-21cyclictest22:28:262
8955996446,12cyclictest352-21runrttasks21:58:392
8955996446,12cyclictest352-21runrttasks21:58:392
8955996442,14cyclictest2374-21systemd-journal00:00:242
895599643,41cyclictest1-21systemd20:28:232
8955996414,12cyclictest6042-21taskset00:18:342
894699644,38cyclictest15827-21latency_hist19:18:261
894699642,21cyclictest360-21fschecks_time00:08:441
895599632,43cyclictest9325-21cat20:38:262
8955996237,17cyclictest0-21swapper/219:38:442
895599623,37cyclictest16443-21fschecks_time20:58:452
895599623,37cyclictest16443-21fschecks_time20:58:452
8955996214,15cyclictest31737-21systemd-run23:13:222
894699623,41cyclictest352-21runrttasks20:04:431
896199614,36cyclictest5781-21chrt23:23:443
896199613,38cyclictest917-21kworker/3:1+events20:58:313
896199613,38cyclictest917-21kworker/3:1+events20:58:313
895599613,31cyclictest11633-21cstates23:33:402
895599613,31cyclictest11633-21cstates23:33:402
896199603,36cyclictest917-21kworker/3:1+events23:08:333
895599602,39cyclictest10263-21timerwakeupswit20:38:592
896199593,40cyclictest32504-21chrt23:13:473
896199584,42cyclictest790-21taskset00:08:553
896199584,38cyclictest917-21kworker/3:1+events19:18:433
896199584,38cyclictest917-21kworker/3:1+events19:18:433
896199583,40cyclictest21691-21taskset23:49:293
896199578,30cyclictest25665-21chrt19:48:273
896199573,41cyclictest12155-21taskset21:45:003
896199573,37cyclictest917-21kworker/3:1+events21:23:573
896199573,37cyclictest12515-21chrt20:48:253
896199573,36cyclictest0-21swapper/323:48:253
896199572,35cyclictest917-21kworker/3:1+events21:28:273
896199563,37cyclictest15180-21chrt22:43:593
896199563,36cyclictest22433-21chrt22:03:093
896199563,36cyclictest22433-21chrt22:03:093
896199563,30cyclictest19372-21taskset21:03:453
896199563,30cyclictest19372-21taskset21:03:453
8961995627,20cyclictest3928-21chrt00:13:573
896199562,34cyclictest917-21kworker/3:1+events22:24:303
896199554,35cyclictest917-21kworker/3:1+events23:33:503
896199554,35cyclictest917-21kworker/3:1+events23:33:503
896199554,34cyclictest6187-21chrt00:18:393
896199553,37cyclictest7642-21chrt21:38:273
896199553,37cyclictest7642-21chrt21:38:273
896199553,34cyclictest8535-21taskset23:28:383
896199552,37cyclictest10676-21chrt00:25:363
896199552,34cyclictest917-21kworker/3:1+events21:33:303
896199545,35cyclictest30869-21taskset21:23:233
896199544,34cyclictest4180-21chrt23:20:323
896199544,34cyclictest4180-21chrt23:20:323
896199543,37cyclictest17322-21chrt23:43:253
896199543,37cyclictest11874-21chrt19:03:593
896199543,36cyclictest28645-21chrt19:58:233
896199543,36cyclictest28645-21chrt19:58:233
896199543,35cyclictest21181-21chrt22:54:013
896199543,34cyclictest18111-21chrt19:23:523
896199542,39cyclictest25853-21taskset23:57:563
8961995420,19cyclictest0-21swapper/321:54:133
896199534,32cyclictest15800-21taskset19:18:253
896199533,35cyclictest22406-21chrt19:38:263
896199533,35cyclictest13700-21chrt19:08:583
896199533,34cyclictest23900-21chrt21:10:423
896199532,37cyclictest5407-21chrt22:28:393
896199524,35cyclictest24414-21chrt22:59:363
896199524,35cyclictest24414-21chrt22:59:363
896199524,33cyclictest28717-21chrt22:13:263
896199523,34cyclictest6860-21chrt20:28:573
896199523,33cyclictest29533-21chrt19:58:563
896199523,33cyclictest29533-21chrt19:58:563
896199523,32cyclictest14698-21taskset20:53:443
896199522,37cyclictest26171-21chrt21:13:593
896199522,37cyclictest26171-21chrt21:13:593
896199513,37cyclictest14565-21chrt21:48:583
896199513,37cyclictest14565-21chrt21:48:583
896199513,36cyclictest28593-21chrt23:08:033
896199513,34cyclictest22951-21chrt19:38:513
896199512,37cyclictest24393-21taskset22:05:153
896199503,36cyclictest19487-21chrt22:52:263
896199503,36cyclictest10451-21chrt19:03:233
896199503,36cyclictest10031-21chrt22:35:563
896199503,35cyclictest32225-21taskset20:08:403
896199503,35cyclictest32225-21taskset20:08:403
896199503,33cyclictest32145-21chrt22:18:463
896199503,33cyclictest1987-21chrt20:13:543
896199503,33cyclictest1987-21chrt20:13:543
896199503,30cyclictest11182-21chrt22:38:323
896199502,34cyclictest28160-21taskset22:12:293
896199493,34cyclictest20136-21chrt19:28:583
896199492,35cyclictest3586-21taskset20:18:563
896199492,35cyclictest3586-21taskset20:18:563
896199484,30cyclictest4562-21taskset20:23:383
896199483,29cyclictest8252-21chrt20:33:513
896199472,33cyclictest30116-21chrt22:14:363
8961994610,23cyclictest12895-21chrt20:48:463
896199453,32cyclictest25389-21chrt19:48:223
896199452,31cyclictest31834-21taskset20:08:033
896199414,27cyclictest10531-21taskset20:40:563
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional