You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-13 - 10:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Fri Mar 13, 2026 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
9323993023,1cyclictest20740-21cpu23:31:510
932699271,0cyclictest331rcuc/323:13:593
932599261,24cyclictest2988-21diskmemload23:44:032
9326992525,0cyclictest331rcuc/323:19:273
9326992523,1cyclictest0-21swapper/322:32:303
9326992523,1cyclictest0-21swapper/321:58:413
9326992522,2cyclictest2988-21diskmemload23:37:003
932699250,24cyclictest2988-21diskmemload23:49:113
9325992523,2cyclictest29303-21apt-get19:51:502
9325992523,2cyclictest27627-21unixbench_multi23:37:002
932599252,22cyclictest0-21swapper/223:21:512
9325992521,2cyclictest2988-21diskmemload21:14:542
932599251,24cyclictest0-21swapper/223:19:272
932499250,24cyclictest18246-21ssh22:57:141
932499250,24cyclictest0-21swapper/100:17:001
932499250,1cyclictest0-21swapper/123:54:451
932399250,24cyclictest0-21swapper/022:16:510
9326992423,1cyclictest0-21swapper/322:05:313
932699241,22cyclictest2015-21kworker/u8:100:35:533
932699241,22cyclictest15724-21sh22:24:263
932699240,23cyclictest679-21ssh22:41:513
932699240,1cyclictest0-21swapper/323:44:033
932699240,0cyclictest0-21swapper/300:28:103
932599241,22cyclictest25638-21ssh22:33:432
932599240,24cyclictest2988-21diskmemload22:45:042
932599240,23cyclictest27134-21ssh22:04:072
932599240,0cyclictest0-21swapper/200:28:102
9324992424,0cyclictest0-21swapper/122:20:231
9324992421,1cyclictest313-21systemd-journal19:20:001
932499240,24cyclictest0-21swapper/122:12:541
932499240,24cyclictest0-21swapper/121:37:261
932499240,23cyclictest0-21swapper/119:21:511
9323992424,0cyclictest2988-21diskmemload21:57:380
9323992422,1cyclictest0-21swapper/022:57:430
9323992421,2cyclictest0-21swapper/000:08:180
932399241,1cyclictest0-21swapper/022:11:270
932399240,2cyclictest8667-21ssh23:51:190
932399240,2cyclictest4472-21cron22:45:000
9326992323,0cyclictest0-21swapper/300:03:513
9326992322,1cyclictest0-21swapper/319:51:503
9326992321,2cyclictest0-21swapper/300:06:103
9326992321,1cyclictest19914-21ssh23:30:413
9326992321,1cyclictest0-21swapper/322:35:443
9326992321,1cyclictest0-21swapper/321:45:243
932699232,0cyclictest10318-21apt-get23:21:513
932699231,21cyclictest0-21swapper/322:57:053
932699230,2cyclictest15520-21which21:21:533
932699230,2cyclictest0-21swapper/321:51:513
932699230,23cyclictest2988-21diskmemload23:52:413
932699230,23cyclictest0-21swapper/323:05:053
932699230,23cyclictest0-21swapper/322:45:043
932699230,23cyclictest0-21swapper/321:44:063
932699230,22cyclictest7516-21ssh00:21:193
932699230,22cyclictest4891-21ssh22:13:413
932699230,22cyclictest2988-21diskmemload22:52:243
932699230,22cyclictest26868-21ssh22:03:433
932699230,22cyclictest0-21swapper/323:02:543
932699230,22cyclictest0-21swapper/321:13:173
932699230,1cyclictest22417-21ssh21:27:533
932699230,1cyclictest0-21swapper/300:32:233
9325992323,0cyclictest2988-21diskmemload22:58:072
9325992323,0cyclictest2988-21diskmemload22:50:142
9325992323,0cyclictest0-21swapper/222:05:312
9325992323,0cyclictest0-21swapper/221:53:032
9325992322,1cyclictest32119-21ssh22:40:142
9325992321,1cyclictest9937-21cstates21:16:532
9325992321,1cyclictest32303-21sh00:12:562
9325992321,1cyclictest18492-21ssh21:24:522
9325992319,3cyclictest22838-21ssh00:35:192
9325992319,3cyclictest0-21swapper/222:10:202
932599231,22cyclictest271rcuc/223:48:312
932599230,2cyclictest19931-21ssh23:30:432
932599230,23cyclictest2988-21diskmemload23:05:052
932599230,23cyclictest0-21swapper/223:29:362
932599230,23cyclictest0-21swapper/223:00:092
932599230,23cyclictest0-21swapper/221:47:592
932599230,23cyclictest0-21swapper/221:26:122
932599230,22cyclictest20001-21diskstats00:01:542
932599230,22cyclictest13632-21sh23:55:532
932599230,22cyclictest0-21swapper/221:55:242
932599230,1cyclictest2988-21diskmemload22:27:292
932599230,1cyclictest2988-21diskmemload00:16:402
932599230,0cyclictest0-21swapper/223:12:572
932599230,0cyclictest0-21swapper/221:44:062
9324992323,0cyclictest2988-21diskmemload23:48:421
9324992323,0cyclictest0-21swapper/123:10:031
9324992323,0cyclictest0-21swapper/122:40:391
9324992323,0cyclictest0-21swapper/122:15:231
9324992323,0cyclictest0-21swapper/100:36:111
9324992323,0cyclictest0-21swapper/100:06:141
9324992321,2cyclictest20564-21ssh23:31:401
932499230,23cyclictest0-21swapper/123:17:001
932499230,23cyclictest0-21swapper/121:23:381
932499230,23cyclictest0-21swapper/121:16:351
932499230,22cyclictest2988-21diskmemload22:48:051
932499230,22cyclictest22158-21ssh21:27:291
932499230,22cyclictest211rcuc/100:22:021
932499230,22cyclictest20898-21ssh21:57:271
932499230,22cyclictest20444-21ssh23:00:141
932499230,22cyclictest15666-21ssh00:28:121
932499230,22cyclictest15465-21/usr/sbin/munin23:26:581
932499230,1cyclictest17200-21ssh23:58:451
932499230,0cyclictest0-21swapper/123:41:231
932499230,0cyclictest0-21swapper/123:20:531
932499230,0cyclictest0-21swapper/122:51:061
932499230,0cyclictest0-21swapper/122:06:091
932499230,0cyclictest0-21swapper/122:00:381
932499230,0cyclictest0-21swapper/100:10:491
9323992323,0cyclictest2988-21diskmemload23:07:080
9323992323,0cyclictest2988-21diskmemload21:18:290
9323992323,0cyclictest0-21swapper/023:57:570
9323992323,0cyclictest0-21swapper/023:35:010
9323992323,0cyclictest0-21swapper/022:36:090
9323992323,0cyclictest0-21swapper/022:31:450
9323992323,0cyclictest0-21swapper/021:33:480
9323992323,0cyclictest0-21swapper/000:33:010
9323992323,0cyclictest0-21swapper/000:21:290
9323992321,1cyclictest2988-21diskmemload23:40:410
9323992321,1cyclictest0-21swapper/021:50:520
9323992321,1cyclictest0-21swapper/021:37:240
9323992320,2cyclictest14114-21cat19:20:000
9323992320,1cyclictest29084-21rm22:06:450
932399230,2cyclictest4871-21ssh21:11:530
932399230,23cyclictest2988-21diskmemload22:22:310
932399230,23cyclictest0-21swapper/022:51:150
932399230,23cyclictest0-21swapper/021:28:040
932399230,22cyclictest622-21gdbus20:00:010
932399230,22cyclictest2988-21diskmemload23:01:470
932399230,22cyclictest2988-21diskmemload22:27:530
932399230,22cyclictest2988-21diskmemload00:25:270
932399230,22cyclictest20547-21dump-pmu-power20:40:010
932399230,22cyclictest20547-21dump-pmu-power20:40:010
932399230,22cyclictest14743-21sh23:25:410
932399230,1cyclictest2988-21diskmemload00:18:590
932399230,1cyclictest0-21swapper/023:11:190
932399230,1cyclictest0-21swapper/000:01:090
932399230,0cyclictest2988-21diskmemload00:37:090
932399230,0cyclictest0-21swapper/022:00:460
9326992222,0cyclictest0-21swapper/322:25:093
9326992222,0cyclictest0-21swapper/300:11:113
9326992220,1cyclictest29461-21sh21:35:313
932699220,22cyclictest0-21swapper/321:30:153
932699220,21cyclictest1357-21perf20:00:013
932699220,21cyclictest0-21swapper/321:15:473
932699220,1cyclictest0-21swapper/323:25:193
932699220,1cyclictest0-21swapper/322:15:103
932699220,1cyclictest0-21swapper/300:16:313
9325992222,0cyclictest271rcuc/221:30:352
9325992222,0cyclictest0-21swapper/222:21:022
9325992221,1cyclictest0-21swapper/223:51:452
9325992220,1cyclictest0-21swapper/222:35:122
932599220,21cyclictest1359-21munin-run20:00:012
932599220,21cyclictest0-21swapper/219:26:502
932599220,21cyclictest0-21swapper/200:21:352
932599220,21cyclictest0-21swapper/200:05:282
932599220,1cyclictest0-21swapper/220:06:512
932599220,1cyclictest0-21swapper/200:30:302
9324992222,0cyclictest0-21swapper/122:30:561
9324992222,0cyclictest0-21swapper/121:12:201
9324992220,2cyclictest6293-21rm21:44:031
932499220,22cyclictest2988-21diskmemload23:07:371
932499220,22cyclictest0-21swapper/121:50:401
932499220,22cyclictest0-21swapper/100:00:111
932499220,21cyclictest17815-21sh00:30:361
932499220,21cyclictest16910-21apt-get19:26:501
932499220,21cyclictest11714-21wc19:15:011
932499220,1cyclictest25592-21ssh23:36:271
932499220,1cyclictest24499-21rm21:30:481
932499220,1cyclictest0-21swapper/122:35:551
932499220,0cyclictest0-21swapper/121:45:521
9323992221,1cyclictest0-21swapper/023:15:150
9323992220,1cyclictest7269-21rm21:45:080
932399220,22cyclictest0-21swapper/021:20:380
932399220,21cyclictest2988-21diskmemload00:10:480
932399220,21cyclictest0-21swapper/021:40:100
932399220,1cyclictest0-21swapper/023:24:000
932399220,0cyclictest0-21swapper/023:45:070
9326992119,1cyclictest15774-21perf20:30:003
9326992119,1cyclictest0-21swapper/321:06:503
9326992119,1cyclictest0-21swapper/320:56:513
932699210,21cyclictest0-21swapper/319:26:503
932699210,20cyclictest0-21swapper/321:01:523
932699210,20cyclictest0-21swapper/319:41:503
932699210,0cyclictest0-21swapper/320:46:513
932699210,0cyclictest0-21swapper/320:06:513
9325992120,0cyclictest26910-21apt-get19:46:522
9325992119,1cyclictest0-21swapper/220:30:002
932599210,21cyclictest0-21swapper/222:15:092
932599210,21cyclictest0-21swapper/220:01:502
932599210,20cyclictest0-21swapper/219:25:002
9324992121,0cyclictest0-21swapper/122:26:111
9324992120,1cyclictest21704-21apt-get19:36:501
932499210,20cyclictest11373-21apt-get20:21:521
932499210,20cyclictest0-21swapper/119:56:511
932499210,0cyclictest30976-21apt-get21:01:521
9323992120,1cyclictest4156-21apt-get20:06:510
9323992119,1cyclictest21325-21munin-run19:35:000
9326992020,0cyclictest0-21swapper/319:16:513
9326992018,1cyclictest0-21swapper/320:15:003
932699200,20cyclictest16526-21perf19:25:003
932699200,0cyclictest0-21swapper/319:31:513
9325992020,0cyclictest0-21swapper/219:16:512
9325992019,1cyclictest0-21swapper/220:46:512
9325992018,1cyclictest8562-21perf20:15:002
9325992017,2cyclictest23731-21sendmail-msp19:40:002
9324992020,0cyclictest0-21swapper/119:31:511
9323992020,0cyclictest0-21swapper/021:01:520
9323992020,0cyclictest0-21swapper/020:26:530
9323992020,0cyclictest0-21swapper/020:01:490
932399200,20cyclictest0-21swapper/019:11:480
9326991919,0cyclictest1755-21apt-get20:01:493
9326991919,0cyclictest0-21swapper/319:46:523
9326991917,1cyclictest23717-21perf19:40:003
9326991916,2cyclictest0-21swapper/319:11:493
932699190,0cyclictest0-21swapper/320:16:523
9325991919,0cyclictest0-21swapper/221:06:502
9325991919,0cyclictest0-21swapper/220:56:512
9324991919,0cyclictest998-21apt-get21:06:501
9324991919,0cyclictest8945-21apt-get20:16:521
9324991919,0cyclictest0-21swapper/120:51:511
9323991919,0cyclictest0-21swapper/020:41:510
9325991816,1cyclictest0-21swapper/219:11:492
9325991815,2cyclictest19743-21pmu-power20:36:582
9325991815,2cyclictest19743-21pmu-power20:36:582
9324991818,0cyclictest0-21swapper/120:31:501
9324991816,1cyclictest15778-21awk20:30:001
9323991818,0cyclictest0-21swapper/019:26:500
9323991816,1cyclictest25385-21munin-run20:50:010
9325991717,0cyclictest19309-21apt-get19:31:512
9325991717,0cyclictest0-21swapper/219:41:502
9325991715,2cyclictest0-21swapper/220:31:502
9325991715,1cyclictest30145-21apt-get21:00:022
9324991715,1cyclictest30162-21awk21:00:001
9323991717,0cyclictest0-21swapper/019:36:500
9323991714,2cyclictest30165-21apt-get21:00:000
9325991611,4cyclictest0-21swapper/220:41:522
9324991616,0cyclictest23373-21apt-get20:46:511
9324991616,0cyclictest0-21swapper/120:06:511
9324991616,0cyclictest0-21swapper/119:51:501
9323991616,0cyclictest0-21swapper/021:06:500
9323991615,1cyclictest0-21swapper/020:31:500
9326991515,0cyclictest0-21swapper/320:31:503
9324991515,0cyclictest0-21swapper/120:11:531
932399150,1cyclictest16564-21cat19:25:000
932699140,14cyclictest0-21swapper/320:50:323
932699140,14cyclictest0-21swapper/320:40:033
932699140,14cyclictest0-21swapper/320:35:083
932699140,14cyclictest0-21swapper/320:35:083
932699140,0cyclictest0-21swapper/320:20:173
932599140,14cyclictest0-21swapper/220:20:202
932599140,0cyclictest0-21swapper/220:50:322
932599140,0cyclictest0-21swapper/220:15:252
9324991414,0cyclictest0-21swapper/120:40:161
9324991414,0cyclictest0-21swapper/120:35:021
9324991414,0cyclictest0-21swapper/120:35:021
932499140,0cyclictest0-21swapper/120:00:531
932499140,0cyclictest0-21swapper/119:40:121
9323991414,0cyclictest0-21swapper/020:20:260
932399140,14cyclictest0-21swapper/020:50:540
932399140,14cyclictest0-21swapper/020:15:070
932399140,14cyclictest0-21swapper/020:11:130
932399140,0cyclictest0-21swapper/019:50:360
932399140,0cyclictest0-21swapper/019:45:160
932399140,0cyclictest0-21swapper/019:40:360
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional