You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-17 - 02:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Thu Sep 17, 2026 00:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
88902239163,25sleep20-21swapper/219:08:152
89132235162,24sleep00-21swapper/019:08:330
88492215144,58sleep10-21swapper/119:07:421
87272203166,24sleep30-21swapper/319:06:073
9125995545,9cyclictest0-21swapper/221:39:212
139752550,3sleep10-21swapper/121:45:131
162482540,3sleep316252-21unixbench_singl00:15:253
302072530,2sleep07-21ksoftirqd/022:20:240
302332510,4sleep230-21rcuc/219:55:142
911299389,13cyclictest19198-21awk20:45:010
226842380,2sleep00-21swapper/020:50:220
222132380,4sleep230-21rcuc/200:30:212
9125993734,2cyclictest0-21swapper/223:31:102
9125993527,3cyclictest0-21swapper/223:37:442
912599340,21cyclictest0-21swapper/221:58:592
912599340,21cyclictest0-21swapper/221:58:592
911299334,13cyclictest25457-21munin-run00:40:010
912599321,15cyclictest0-21swapper/223:52:432
912599321,15cyclictest0-21swapper/222:20:212
911999321,16cyclictest0-21swapper/120:16:421
911299321,15cyclictest24206-21munin-run22:10:010
912599311,13cyclictest0-21swapper/222:11:272
912599310,21cyclictest0-21swapper/222:44:552
911299316,12cyclictest21078-21perf00:30:010
912599301,13cyclictest0-21swapper/222:26:472
912599301,13cyclictest0-21swapper/221:25:192
912599300,18cyclictest0-21swapper/221:20:472
911299302,13cyclictest5033-21munin-run22:40:010
912599291,13cyclictest0-21swapper/221:19:272
912599291,13cyclictest0-21swapper/219:41:152
912599290,11cyclictest0-21swapper/223:28:212
911299292,15cyclictest16038-21munin-run23:05:010
912599285,9cyclictest23782-21sed23:20:172
912599281,13cyclictest0-21swapper/223:01:032
912599281,12cyclictest0-21swapper/223:49:272
912599281,11cyclictest0-21swapper/219:33:412
911999288,18cyclictest0-21swapper/100:10:021
9119992813,12cyclictest22627-21munin-run23:20:011
911299282,12cyclictest3463-21perf23:50:010
251212280,4sleep07-21ksoftirqd/022:10:160
912599279,3cyclictest16770-21tr21:50:212
912599271,9cyclictest0-21swapper/219:25:172
912599271,18cyclictest0-21swapper/221:42:232
912599271,17cyclictest0-21swapper/220:27:392
911999271,9cyclictest8365-21memory22:45:191
911299273,11cyclictest0-21swapper/021:20:010
82422270,1sleep00-21swapper/000:00:130
912599263,9cyclictest22427-40dnf23:17:432
9125992623,2cyclictest0-21swapper/221:06:432
912599261,3cyclictest0-21swapper/200:20:232
912599261,3cyclictest0-21swapper/200:20:232
912599261,11cyclictest0-21swapper/220:55:572
912599260,9cyclictest0-21swapper/200:25:212
912599260,11cyclictest0-21swapper/219:45:132
912599260,10cyclictest0-21swapper/220:40:132
912599260,10cyclictest0-21swapper/219:10:332
912599251,13cyclictest0-21swapper/220:52:032
912599251,12cyclictest0-21swapper/220:49:152
912599251,12cyclictest0-21swapper/220:49:152
912599250,15cyclictest17108-21latency_hist20:40:022
911299253,11cyclictest1830-21/usr/sbin/munin20:05:140
260782250,3sleep341-21ksoftirqd/323:25:213
912599249,3cyclictest0-21swapper/200:01:032
912599241,9cyclictest0-21swapper/219:21:052
912599241,14cyclictest0-21swapper/222:57:282
912599241,13cyclictest0-21swapper/223:58:072
912599241,11cyclictest0-21swapper/222:04:552
912599241,10cyclictest0-21swapper/220:10:512
912599240,12cyclictest0-21swapper/222:06:112
911999247,9cyclictest2-21kthreadd00:30:211
9119992419,3cyclictest0-21swapper/121:55:141
9119992419,3cyclictest0-21swapper/121:55:141
911999241,17cyclictest0-21swapper/121:12:261
9119992410,5cyclictest19190-21awk20:45:011
911299248,6cyclictest0-21swapper/023:24:470
911299240,11cyclictest17619-21perf21:55:010
913099232,8cyclictest0-21swapper/323:17:423
912599231,9cyclictest0-21swapper/220:22:032
912599231,2cyclictest0-21swapper/222:45:112
912599231,13cyclictest0-21swapper/200:19:552
912599231,12cyclictest0-21swapper/221:49:592
912599230,2cyclictest0-21swapper/223:05:272
912599230,2cyclictest0-21swapper/223:05:272
911999239,12cyclictest19766-21kworker/u8:222:33:511
911999231,19cyclictest25777-21munin-run21:00:011
9119992311,5cyclictest0-21swapper/122:56:261
911999230,17cyclictest0-21swapper/123:23:441
154212230,4sleep339-21rcuc/321:50:013
94842221,4sleep19502-21ntp_states20:20:221
913099229,3cyclictest32237-21grep23:40:153
913099221,12cyclictest7452-21/usr/sbin/munin22:45:143
912599229,3cyclictest18457-21runrttasks23:10:132
912599229,3cyclictest0-21swapper/220:01:212
912599227,2cyclictest0-21swapper/219:35:212
912599225,3cyclictest0-21swapper/222:35:172
912599221,8cyclictest0-21swapper/219:18:472
912599221,14cyclictest0-21swapper/221:00:232
912599220,10cyclictest0-21swapper/220:05:152
911999222,11cyclictest6317-21ntp_states22:40:191
911999221,9cyclictest98392cut19:10:141
911999221,3cyclictest9992-21df22:50:131
911999221,19cyclictest0-21swapper/100:25:401
911999221,18cyclictest19951-21tr19:30:221
911999221,15cyclictest0-21swapper/121:22:261
911299226,2cyclictest26042-21kworker/0:223:29:450
913099217,3cyclictest39-21rcuc/319:30:263
913099215,4cyclictest2670-21kernelversion20:05:173
913099211,5cyclictest0-21swapper/322:05:033
913099211,11cyclictest28765-21tune2fs21:05:173
912599217,2cyclictest0-21swapper/220:30:392
912599211,5cyclictest0-21swapper/200:13:352
912599211,2cyclictest87450irq/127-enp1s0-19:51:012
912599211,2cyclictest0-21swapper/220:18:232
912599210,9cyclictest0-21swapper/200:36:212
911999217,6cyclictest0-21swapper/123:47:121
911999212,3cyclictest0-21swapper/123:11:231
911999212,3cyclictest0-21swapper/122:13:051
911999212,17cyclictest0-21swapper/123:43:561
911999212,17cyclictest0-21swapper/119:58:501
9119992115,3cyclictest13809-21vmstat19:15:271
911999211,2cyclictest0-21swapper/122:15:431
9119992112,2cyclictest0-21swapper/121:32:161
911999211,18cyclictest0-21swapper/122:06:431
911999211,18cyclictest0-21swapper/120:00:241
911999211,14cyclictest0-21swapper/121:41:251
911999211,14cyclictest0-21swapper/121:37:281
911999211,11cyclictest21435-21cat20:50:011
911999211,11cyclictest21435-21cat20:50:011
911999211,10cyclictest23584-21perf20:55:011
911999211,10cyclictest0-21swapper/123:01:151
911299216,5cyclictest0-21swapper/021:34:040
911299214,6cyclictest7919-40dnf20:16:410
911299211,11cyclictest28883-21cat19:50:250
911299211,10cyclictest11532-21fgrep00:05:270
264692211,3sleep126673-21smartctl21:00:151
101752210,3sleep011-21rcuc/019:10:170
913099208,3cyclictest10295-21/usr/sbin/munin00:05:243
913099202,8cyclictest423950kworker/u8:021:50:153
913099201,8cyclictest302082cat22:20:243
913099201,6cyclictest5080-21timerwakeupswit23:50:243
913099201,5cyclictest543-21perf22:30:013
9130992013,4cyclictest22145-21fschecks_time20:50:163
913099201,16cyclictest5869-21/usr/sbin/munin23:55:113
913099201,13cyclictest0-21swapper/322:50:243
913099201,11cyclictest839-21lldpd20:42:373
913099201,11cyclictest26001-21/usr/sbin/munin21:00:153
913099201,11cyclictest25393-21/usr/sbin/munin19:45:253
913099201,10cyclictest31981-21/usr/sbin/munin20:00:173
912599201,11cyclictest32447-21irqstats23:40:172
911999208,4cyclictest17845-21users23:05:261
911999208,4cyclictest17845-21users23:05:261
911999202,3cyclictest0-21swapper/121:09:491
911999202,17cyclictest0-21swapper/122:04:541
911999202,16cyclictest0-21swapper/123:26:381
911999202,16cyclictest0-21swapper/119:39:411
911999201,3cyclictest0-21swapper/123:55:221
911999201,3cyclictest0-21swapper/121:16:391
911999201,17cyclictest0-21swapper/122:39:081
911999201,17cyclictest0-21swapper/120:31:211
911999201,17cyclictest0-21swapper/119:27:361
911999201,11cyclictest4344-21expr20:10:131
911999201,10cyclictest2149-21expr20:05:131
911999201,10cyclictest16546-21memory21:50:181
911999201,10cyclictest16371-21latency_hist19:25:011
911999200,18cyclictest0-21swapper/119:50:221
911999200,14cyclictest0-21swapper/123:30:421
911299209,3cyclictest17829-21/usr/sbin/munin21:55:170
911299209,3cyclictest17829-21/usr/sbin/munin21:55:170
911299206,5cyclictest0-21swapper/021:24:400
911299202,10cyclictest3672-21/usr/sbin/munin23:50:210
911299202,10cyclictest31022-21/usr/sbin/munin22:25:230
911299201,6cyclictest0-21swapper/023:30:340
911299201,3cyclictest0-21swapper/023:07:530
911299201,3cyclictest0-21swapper/023:07:530
911299201,13cyclictest0-21swapper/022:30:130
911299201,11cyclictest28213-21/usr/sbin/munin21:05:180
911299201,11cyclictest23808-21/usr/sbin/munin20:55:170
911299201,11cyclictest21291-21/usr/sbin/munin00:30:270
911299201,11cyclictest11238-21/usr/sbin/munin21:40:280
911299201,11cyclictest0-21swapper/023:36:370
322082201,3sleep20-21swapper/221:13:012
913099194,6cyclictest808-21snmpd00:23:283
913099194,6cyclictest808-21snmpd00:23:283
913099193,8cyclictest87350irq/126-enp1s0-23:00:173
913099192,4cyclictest8219-21cpuspeed_turbos00:00:133
913099191,9cyclictest4822-21taskset22:37:303
913099191,8cyclictest24855-21kworker/3:119:25:233
913099191,11cyclictest0-21swapper/321:25:083
913099190,9cyclictest23301-21cat00:35:013
911999195,3cyclictest162512grep00:15:251
9119991916,2cyclictest0-21swapper/123:54:101
9119991916,2cyclictest0-21swapper/120:39:151
9119991916,2cyclictest0-21swapper/119:48:221
9119991915,3cyclictest0-21swapper/122:27:131
9119991915,3cyclictest0-21swapper/120:25:091
9119991915,3cyclictest0-21swapper/119:44:511
911999191,17cyclictest0-21swapper/122:22:261
911999191,17cyclictest0-21swapper/100:36:551
911999191,12cyclictest0-21swapper/123:36:501
911999191,10cyclictest17643-21if_enp2s000:20:171
911999191,10cyclictest17643-21if_enp2s000:20:171
911999190,2cyclictest87650irq/129-enp1s0-21:25:041
911299198,10cyclictest0-21swapper/019:25:010
911299196,5cyclictest0-21swapper/022:03:090
911299193,7cyclictest8422-21/usr/sbin/munin20:20:170
911299191,5cyclictest6245-21needreboot22:40:180
911299191,5cyclictest0-21swapper/022:49:380
911299191,11cyclictest0-21swapper/021:48:520
911299191,10cyclictest9037-21/usr/sbin/munin21:35:240
911299191,10cyclictest30178-21diskmemload23:40:160
911299191,10cyclictest30178-21diskmemload22:50:220
911299191,10cyclictest0-21swapper/022:57:480
911299191,10cyclictest0-21swapper/019:46:390
69232191,2sleep30-21swapper/322:41:193
121422191,3sleep312204-21diskstats22:55:153
913099187,3cyclictest87650irq/129-enp1s0-21:17:303
913099186,3cyclictest24748-21grep19:40:243
913099185,3cyclictest87650irq/129-enp1s0-21:41:243
913099185,3cyclictest21172-21pmu-power22:00:213
913099182,7cyclictest4025-21/usr/sbin/munin20:10:253
913099182,4cyclictest28588-21needreboot19:50:203
913099182,4cyclictest12788-21perl19:15:163
913099181,7cyclictest39-21rcuc/323:05:253
913099181,7cyclictest39-21rcuc/323:05:253
913099181,6cyclictest0-21swapper/321:32:213
913099181,5cyclictest25347-21cut22:10:193
913099181,5cyclictest0-21swapper/323:49:233
913099181,5cyclictest0-21swapper/323:10:243
913099181,4cyclictest8422-21/usr/sbin/munin20:20:253
9130991812,3cyclictest827-21runrttasks23:20:253
9130991812,3cyclictest31370-21users19:55:253
9130991812,3cyclictest30746-21df21:10:143
913099181,10cyclictest839-21lldpd00:26:403
912599185,3cyclictest0-21swapper/222:30:172
912599180,9cyclictest0-21swapper/221:31:012
911299188,3cyclictest20994-21/usr/sbin/munin19:35:200
911299181,9cyclictest579-21systemd-journal23:55:400
911299181,6cyclictest0-21swapper/022:19:410
911299181,5cyclictest4642-21/usr/sbin/munin21:25:240
911299181,5cyclictest20653-21/usr/sbin/munin23:15:180
911299181,5cyclictest18798-21/usr/sbin/munin19:30:120
911299181,10cyclictest0-21swapper/023:11:410
911299180,12cyclictest0-21swapper/000:15:180
115382181,2sleep20-21swapper/222:54:022
913099176,3cyclictest87350irq/126-enp1s0-00:38:203
913099175,3cyclictest12050irq/123-ahci[0023:35:023
913099171,9cyclictest0-21swapper/319:35:173
913099171,9cyclictest0-21swapper/300:11:493
913099171,7cyclictest12822-21/usr/sbin/munin20:30:233
913099171,6cyclictest27188-21perl22:15:193
913099171,5cyclictest0-21swapper/322:30:143
913099171,3cyclictest3788-21cat21:20:233
913099171,2cyclictest0-21swapper/321:35:173
9130991711,3cyclictest6245-21cpu20:15:153
9130991711,3cyclictest18178-21diskstats21:55:133
9130991711,3cyclictest18178-21diskstats21:55:133
9130991711,3cyclictest10626-21/usr/sbin/munin20:25:113
913099171,10cyclictest0-21swapper/320:37:073
912599176,3cyclictest0-21swapper/200:09:592
912599170,11cyclictest0-21swapper/222:16:592
9119991711,4cyclictest8722-21tail00:00:161
9119991711,3cyclictest11363-21ntp_states00:05:251
911299179,5cyclictest0-21swapper/020:25:250
911299171,9cyclictest0-21swapper/000:10:190
911299171,7cyclictest12210-21/usr/sbin/munin19:15:250
9112991712,3cyclictest29785-21/usr/sbin/munin19:55:240
9112991711,3cyclictest31062-21tail21:10:160
9112991711,3cyclictest16897-21/usr/sbin/munin00:20:180
9112991711,3cyclictest16897-21/usr/sbin/munin00:20:180
210362171,2sleep00-21swapper/020:45:460
210362171,2sleep00-21swapper/020:45:460
913099165,3cyclictest87650irq/129-enp1s0-19:20:153
913099161,6cyclictest30150-21basename23:35:173
913099161,6cyclictest0-21swapper/320:56:233
913099161,12cyclictest20474-21ntp_states20:45:223
913099161,12cyclictest20474-21ntp_states20:45:223
913099160,14cyclictest0-21swapper/319:11:553
911299164,3cyclictest4089-21cpuspeed_turbos20:10:120
911299161,9cyclictest0-21swapper/020:00:180
911299161,6cyclictest26813-21/usr/sbin/munin21:00:170
911299161,6cyclictest0-21swapper/019:40:170
911299161,14cyclictest0-21swapper/020:34:030
911299161,13cyclictest18452-21sshd19:28:400
911299161,12cyclictest15018-21/usr/sbin/munin20:35:190
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional