You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-25 - 12:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Thu Jun 25, 2026 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2768099260,25cyclictest26798-21ssh21:14:492
2767599260,25cyclictest16722-21awk23:00:001
27682992522,2cyclictest22017-21ssh23:31:593
27680992522,2cyclictest303-21ssh21:47:422
2768099250,24cyclictest21561-21apt-get22:35:322
27671992523,1cyclictest24190-21cron23:35:000
2767199250,24cyclictest6725-21ssh23:19:120
2767199250,24cyclictest0-21swapper/021:49:220
27682992423,1cyclictest0-21swapper/322:23:093
2768299241,22cyclictest1514-21munin-run00:40:013
2768299240,23cyclictest21342-21diskmemload00:05:143
2768299240,23cyclictest20359-21ssh22:05:343
2768299240,23cyclictest15927-21ssh22:30:323
27680992420,4cyclictest0-21swapper/223:17:172
2768099240,23cyclictest21342-21diskmemload23:04:302
2768099240,23cyclictest0-21swapper/200:01:382
27675992421,2cyclictest19694-21sh23:30:341
2767599240,24cyclictest0-21swapper/122:53:431
2767599240,23cyclictest75-21kswapd021:45:351
2767599240,23cyclictest6135-21taskset21:52:521
2767599240,23cyclictest21342-21diskmemload00:08:121
2767599240,23cyclictest0-21swapper/123:35:331
2767599240,23cyclictest0-21swapper/123:25:071
2767599240,23cyclictest0-21swapper/122:25:211
2767599240,1cyclictest9053-21cron23:50:011
27671992422,1cyclictest62550irq/127-eno123:39:460
27671992421,2cyclictest31453-21ssh21:18:350
27671992421,2cyclictest13348-21/usr/sbin/munin21:30:390
27671992421,2cyclictest13348-21/usr/sbin/munin21:30:390
2767199240,24cyclictest0-21swapper/022:39:540
2767199240,23cyclictest0-21swapper/021:51:430
2767199240,23cyclictest0-21swapper/000:17:550
2767199240,23cyclictest0-21swapper/000:11:290
27682992323,0cyclictest21342-21diskmemload23:58:403
27682992323,0cyclictest0-21swapper/323:54:553
27682992323,0cyclictest0-21swapper/321:37:573
27682992322,1cyclictest21342-21diskmemload22:45:013
27682992321,1cyclictest4157-21apt-get23:45:343
27682992321,1cyclictest0-21swapper/320:15:013
2768299231,1cyclictest21342-21diskmemload00:29:033
2768299230,22cyclictest6546-21expr22:50:353
2768299230,22cyclictest5305-21rm21:24:153
2768299230,22cyclictest4702-21ssh21:50:473
2768299230,22cyclictest31127-21sh22:15:143
2768299230,22cyclictest25430-21ssh22:38:293
2768299230,22cyclictest24680-21apt-get22:10:023
2768299230,22cyclictest2036-21latency_hist21:50:013
2768299230,22cyclictest0-21swapper/323:17:573
2768299230,1cyclictest21342-21diskmemload23:27:203
2768299230,1cyclictest21342-21diskmemload00:21:243
2768299230,1cyclictest12914-21diskmemload22:26:443
2768299230,1cyclictest0-21swapper/323:05:013
2768299230,0cyclictest21342-21diskmemload21:32:113
2768299230,0cyclictest21342-21diskmemload21:32:113
2768299230,0cyclictest21342-21diskmemload21:16:063
2768299230,0cyclictest21342-21diskmemload00:10:263
2768299230,0cyclictest0-21swapper/322:40:573
27680992323,0cyclictest0-21swapper/223:57:042
27680992323,0cyclictest0-21swapper/223:23:282
27680992323,0cyclictest0-21swapper/222:00:592
27680992323,0cyclictest0-21swapper/221:51:252
27680992323,0cyclictest0-21swapper/200:21:522
27680992322,1cyclictest12917-21ssh23:52:342
27680992321,1cyclictest0-21swapper/221:15:052
27680992320,1cyclictest23332-21ssh23:05:282
2768099230,2cyclictest17690-21ssh21:35:232
2768099230,23cyclictest21342-21diskmemload22:44:372
2768099230,23cyclictest21342-21diskmemload00:06:502
2768099230,23cyclictest0-21swapper/222:50:022
2768099230,22cyclictest9358-21ssh21:27:062
2768099230,22cyclictest3594-21ssh00:12:452
2768099230,22cyclictest1983-21ssh23:43:062
2768099230,22cyclictest15780-21ssh22:30:232
2768099230,22cyclictest14586-21sh21:31:322
2768099230,22cyclictest14586-21sh21:31:322
2768099230,22cyclictest10101-21sh22:25:172
2768099230,22cyclictest0-21swapper/223:33:012
2768099230,22cyclictest0-21swapper/200:26:372
2768099230,1cyclictest2044-21ssh22:17:242
2768099230,0cyclictest21342-21diskmemload21:41:242
2768099230,0cyclictest0-21swapper/223:46:562
2768099230,0cyclictest0-21swapper/223:27:132
2768099230,0cyclictest0-21swapper/222:07:002
2768099230,0cyclictest0-21swapper/200:15:172
27675992323,0cyclictest0-21swapper/122:08:321
27675992323,0cyclictest0-21swapper/121:26:101
27675992322,1cyclictest0-21swapper/122:20:371
27675992321,1cyclictest22122-21munin-run20:05:011
27675992321,1cyclictest0-21swapper/123:50:381
27675992321,1cyclictest0-21swapper/121:45:011
2767599230,2cyclictest7206-21awk00:15:421
2767599230,2cyclictest23955-21sh22:36:231
2767599230,22cyclictest4649-21ssh00:14:091
2767599230,22cyclictest30030-21ssh21:16:301
2767599230,22cyclictest28868-21ssh00:35:321
2767599230,22cyclictest21342-21diskmemload23:55:351
2767599230,22cyclictest19670-21rm21:35:501
2767599230,22cyclictest17983-21sh22:03:281
2767599230,22cyclictest11364-21sh00:20:161
2767599230,22cyclictest0-21swapper/122:33:191
2767599230,22cyclictest0-21swapper/121:30:321
2767599230,22cyclictest0-21swapper/121:30:321
2767599230,22cyclictest0-21swapper/100:01:551
2767599230,0cyclictest21342-21diskmemload00:28:481
2767599230,0cyclictest0-21swapper/121:24:121
2767599230,0cyclictest0-21swapper/121:13:361
27671992323,0cyclictest21342-21diskmemload23:56:300
27671992323,0cyclictest0-21swapper/023:48:220
27671992323,0cyclictest0-21swapper/023:00:280
27671992323,0cyclictest0-21swapper/022:56:070
27671992323,0cyclictest0-21swapper/022:03:360
27671992323,0cyclictest0-21swapper/021:10:070
2767199232,21cyclictest62550irq/127-eno121:56:300
27671992321,1cyclictest7990-21apt-get23:20:320
27671992321,1cyclictest27789-21fschecks_time22:40:340
27671992321,1cyclictest24714-21processes23:05:390
27671992321,1cyclictest11376-21basename20:50:010
27671992321,1cyclictest11376-21basename20:50:010
27671992321,1cyclictest0-21swapper/023:26:320
27671992321,1cyclictest0-21swapper/022:25:020
27671992321,1cyclictest0-21swapper/021:21:440
2767199230,23cyclictest21342-21diskmemload22:30:500
2767199230,23cyclictest0-21swapper/023:51:110
2767199230,23cyclictest0-21swapper/000:27:540
2767199230,22cyclictest30269-21cmp23:40:010
2767199230,22cyclictest0-21swapper/000:35:280
2767199230,0cyclictest0-21swapper/022:50:070
2767199230,0cyclictest0-21swapper/022:17:180
2767199230,0cyclictest0-21swapper/000:33:050
2767199230,0cyclictest0-21swapper/000:01:100
27682992221,1cyclictest0-21swapper/322:55:143
27682992221,1cyclictest0-21swapper/322:00:043
27682992221,0cyclictest0-21swapper/323:35:343
27682992220,1cyclictest13754-21grep20:55:013
27682992220,1cyclictest0-21swapper/321:11:373
27682992220,1cyclictest0-21swapper/300:15:263
27682992220,1cyclictest0-21swapper/300:00:343
2768299220,2cyclictest10873-21sh23:22:093
2768299220,21cyclictest30415-21ssh23:40:023
2768299220,21cyclictest10015-21apt-get19:40:323
2768299220,21cyclictest0-21swapper/323:10:143
2768299220,21cyclictest0-21swapper/300:30:363
2768299220,1cyclictest16916-21munin-run19:55:003
2768299220,1cyclictest0-21swapper/323:00:363
2768299220,1cyclictest0-21swapper/321:25:323
2768299220,0cyclictest0-21swapper/321:40:333
27680992222,0cyclictest0-21swapper/222:20:142
27680992222,0cyclictest0-21swapper/221:56:232
27680992220,1cyclictest6565-21sendmail-msp20:40:002
27680992220,1cyclictest1285-21irqstats22:45:362
2768099220,21cyclictest22405-21apt-get20:05:322
2768099220,21cyclictest21342-21diskmemload22:10:032
2768099220,21cyclictest21342-21diskmemload00:36:052
2768099220,21cyclictest0-21swapper/219:25:322
2768099220,1cyclictest23697-21diskstats00:30:352
2768099220,1cyclictest21342-21diskmemload22:55:332
2768099220,1cyclictest0-21swapper/223:35:072
2768099220,1cyclictest0-21swapper/223:10:152
2768099220,1cyclictest0-21swapper/221:24:452
27675992222,0cyclictest21342-21diskmemload23:11:351
27675992222,0cyclictest0-21swapper/123:20:211
27675992222,0cyclictest0-21swapper/122:15:241
27675992222,0cyclictest0-21swapper/121:57:371
27675992220,1cyclictest606-21dbus-daemon19:15:011
27675992220,1cyclictest4906-21munin-run19:30:011
27675992220,1cyclictest13770-21munin-run20:55:011
2767599220,21cyclictest23247-21apt-get00:30:331
2767599220,21cyclictest14051-21apt-get20:55:311
2767599220,21cyclictest0-21swapper/123:15:461
2767599220,21cyclictest0-21swapper/123:05:301
2767599220,21cyclictest0-21swapper/122:45:411
2767599220,1cyclictest0-21swapper/123:01:341
2767599220,1cyclictest0-21swapper/122:40:341
2767599220,0cyclictest0-21swapper/122:10:021
27671992222,0cyclictest0-21swapper/021:25:030
27671992221,0cyclictest0-21swapper/021:05:020
27671992220,1cyclictest32765-21apt-get19:20:310
2767199220,22cyclictest62550irq/127-eno122:22:090
2767199220,22cyclictest0-21swapper/021:35:450
2767199220,21cyclictest4145-21turbostat.cron20:35:000
2767199220,1cyclictest0-21swapper/022:10:220
2767199220,1cyclictest0-21swapper/021:40:440
2767199220,1cyclictest0-21swapper/020:20:340
2767199220,0cyclictest21342-21diskmemload00:21:270
27682992121,0cyclictest0-21swapper/321:56:583
27682992119,1cyclictest622-21gdbus21:00:013
2768299210,20cyclictest2003-21apt-get20:30:343
2768299210,20cyclictest0-21swapper/320:15:343
27680992120,1cyclictest17595-21apt-get19:55:332
2768099210,20cyclictest27955-21apt-get19:10:322
2768099210,20cyclictest0-21swapper/219:20:312
2768099210,0cyclictest5202-21apt-get19:30:332
27675992121,0cyclictest0-21swapper/123:40:341
27675992119,1cyclictest12126-21munin-run19:45:001
27675992119,1cyclictest0-21swapper/121:10:001
2767599210,20cyclictest4451-21apt-get20:35:321
2767599210,0cyclictest0-21swapper/120:30:331
2767599210,0cyclictest0-21swapper/119:20:321
27671992121,0cyclictest0-21swapper/020:15:340
27671992119,1cyclictest0-21swapper/019:35:340
2767199210,21cyclictest0-21swapper/020:10:010
2767199210,20cyclictest0-21swapper/020:50:330
27682992020,0cyclictest0-21swapper/319:55:023
27682992020,0cyclictest0-21swapper/319:10:323
27682992018,1cyclictest24500-21wc20:10:013
2768299200,0cyclictest0-21swapper/319:30:333
27680992020,0cyclictest0-21swapper/221:00:332
27680992018,1cyclictest21342-21diskmemload21:10:002
27675992020,0cyclictest0-21swapper/120:05:321
27675992019,1cyclictest14804-21apt-get19:50:331
27671992020,0cyclictest0-21swapper/019:55:330
27682991919,0cyclictest0-21swapper/321:00:333
2768299190,18cyclictest0-21swapper/320:00:333
2768299190,0cyclictest0-21swapper/320:45:343
2768299190,0cyclictest0-21swapper/320:45:343
27680991917,1cyclictest12107-21apt19:45:002
27675991919,0cyclictest14509-21perf19:50:001
27675991918,1cyclictest24800-21apt-get20:10:331
27671991917,1cyclictest0-21swapper/019:20:010
27682991818,0cyclictest0-21swapper/320:40:333
27682991815,2cyclictest4894-21dump-pmu-power19:30:013
27680991818,0cyclictest0-21swapper/220:55:312
27680991818,0cyclictest0-21swapper/220:45:342
27680991818,0cyclictest0-21swapper/220:45:342
27680991818,0cyclictest0-21swapper/220:15:342
27671991818,0cyclictest0-21swapper/021:00:330
27671991818,0cyclictest0-21swapper/020:40:330
27671991818,0cyclictest0-21swapper/019:50:330
27671991817,1cyclictest1688-21turbostat20:30:000
27671991815,2cyclictest12112-21grep19:45:000
2767199180,17cyclictest0-21swapper/019:45:320
27680991717,0cyclictest0-21swapper/219:45:322
27680991714,2cyclictest32475-21sendmail-msp19:20:012
27675991717,0cyclictest0-21swapper/119:30:331
27675991715,1cyclictest29313-21latency_hist20:20:001
27671991715,1cyclictest26894-21turbostat20:15:010
27682991616,0cyclictest0-21swapper/320:20:343
27682991615,0cyclictest0-21swapper/320:25:313
27682991614,1cyclictest0-21swapper/319:45:373
27680991616,0cyclictest0-21swapper/220:25:322
2768099160,15cyclictest16966-21munin-run19:55:012
2767599160,15cyclictest0-21swapper/120:45:011
2767199160,15cyclictest21334-21awk20:00:400
2767199160,15cyclictest0-21swapper/019:10:330
27682991513,1cyclictest619-21polkitd20:40:003
2768299150,15cyclictest1601-21ps19:20:383
27680991515,0cyclictest7599-21apt-get19:35:342
27680991514,1cyclictest29624-21apt-get20:20:342
27675991515,0cyclictest0-21swapper/120:50:011
27675991515,0cyclictest0-21swapper/120:50:011
2767599150,5cyclictest0-21swapper/121:00:331
2767599150,14cyclictest0-21swapper/120:20:381
27671991514,0cyclictest2751-21apt-get19:25:320
2768299140,0cyclictest0-21swapper/321:05:033
2768299140,0cyclictest0-21swapper/319:35:343
2768299140,0cyclictest0-21swapper/319:17:593
27680991414,0cyclictest0-21swapper/220:52:182
2768099140,14cyclictest0-21swapper/220:43:092
2768099140,14cyclictest0-21swapper/220:31:032
2768099140,0cyclictest0-21swapper/220:10:072
2768099140,0cyclictest0-21swapper/220:00:392
27675991414,0cyclictest0-21swapper/119:55:451
27675991412,1cyclictest0-21swapper/119:15:391
2767599140,0cyclictest0-21swapper/120:25:361
2767199140,14cyclictest0-21swapper/020:55:120
2767199140,14cyclictest0-21swapper/020:35:500
2767199140,14cyclictest0-21swapper/019:30:470
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional