You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-25 - 00:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Fri Jul 24, 2026 12:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
94142236162,26sleep00-21swapper/007:07:340
92822232162,20sleep20-21swapper/207:05:512
93182207132,24sleep10-21swapper/107:06:191
76382205169,23sleep30-21swapper/307:05:053
1245821720,3sleep0969099cyclictest09:40:160
9701996359,3cyclictest0-21swapper/207:47:322
165102540,5sleep1969599cyclictest09:50:131
323062480,2sleep121-21rcuc/110:25:161
9701994441,2cyclictest0-21swapper/209:52:192
970699391,27cyclictest30756-21diskmemload09:17:133
969099399,14cyclictest21343-21munin-run07:35:000
970199380,28cyclictest0-21swapper/212:11:072
970699351,19cyclictest0-21swapper/312:10:063
969099357,12cyclictest1849-21idleruntime-cro11:45:010
970699331,30cyclictest0-21swapper/308:04:503
970699331,21cyclictest0-21swapper/308:55:293
970699331,14cyclictest0-21swapper/310:53:413
969099336,12cyclictest18820-21awk11:10:000
970699321,22cyclictest0-21swapper/309:01:133
970699320,22cyclictest0-21swapper/309:07:493
970699320,22cyclictest0-21swapper/309:07:493
970699311,23cyclictest0-21swapper/310:50:013
970699311,20cyclictest0-21swapper/308:41:373
970699311,20cyclictest0-21swapper/308:22:133
970699301,13cyclictest0-21swapper/309:27:213
970699301,12cyclictest0-21swapper/309:43:023
970699300,21cyclictest0-21swapper/307:16:453
9695993015,6cyclictest0-21swapper/110:03:211
970699291,12cyclictest0-21swapper/311:05:533
970699291,12cyclictest0-21swapper/309:14:453
970699290,12cyclictest0-21swapper/307:33:373
970199290,11cyclictest0-21swapper/212:08:152
9690992910,16cyclictest24159-21munin-run08:55:010
970699289,8cyclictest0-21swapper/307:50:183
9706992817,3cyclictest0-21swapper/307:28:053
970699281,13cyclictest0-21swapper/311:18:333
970699281,13cyclictest0-21swapper/308:53:533
970699281,12cyclictest0-21swapper/312:01:063
970699281,12cyclictest0-21swapper/311:58:223
970699281,12cyclictest0-21swapper/310:20:423
970699280,13cyclictest0-21swapper/311:50:163
970699280,11cyclictest0-21swapper/309:59:543
969099289,7cyclictest22841-21kworker/0:111:37:110
9706992716,3cyclictest0-21swapper/308:31:253
970699271,14cyclictest0-21swapper/310:08:063
970699271,14cyclictest0-21swapper/309:52:183
970699271,13cyclictest0-21swapper/310:01:583
970699271,12cyclictest0-21swapper/311:24:493
970699271,11cyclictest0-21swapper/310:30:053
9706992710,8cyclictest192512sleep312:22:583
970699270,18cyclictest0-21swapper/307:37:053
970699270,11cyclictest0-21swapper/309:31:253
970699270,10cyclictest0-21swapper/307:55:383
970699270,10cyclictest0-21swapper/307:55:383
969599271,12cyclictest27251-21hddtemp_smartct09:00:161
970699268,3cyclictest14352-21df_inode09:45:143
970699266,9cyclictest0-21swapper/307:10:093
970699261,13cyclictest0-21swapper/312:32:533
970699261,12cyclictest0-21swapper/308:37:173
970699261,12cyclictest0-21swapper/308:26:413
970699261,12cyclictest0-21swapper/308:15:503
970699261,12cyclictest0-21swapper/307:42:453
970699261,11cyclictest0-21swapper/311:36:213
970699261,11cyclictest0-21swapper/308:46:413
970699261,10cyclictest0-21swapper/310:15:543
970699261,10cyclictest0-21swapper/310:15:543
970699260,9cyclictest0-21swapper/308:10:283
970699260,12cyclictest0-21swapper/311:11:133
970699260,11cyclictest0-21swapper/311:33:333
970699260,10cyclictest0-21swapper/309:21:053
970699260,10cyclictest0-21swapper/307:48:103
970199269,8cyclictest0-21swapper/211:20:182
9695992616,6cyclictest4814-21processes10:35:211
969599261,12cyclictest0-21swapper/109:41:221
9695992611,12cyclictest6139-21kworker/u8:008:24:331
970699251,15cyclictest0-21swapper/311:26:573
970699251,15cyclictest0-21swapper/311:26:573
970699251,11cyclictest0-21swapper/311:47:063
970699250,11cyclictest0-21swapper/312:37:573
9695992512,10cyclictest21337-21latency_hist07:35:011
969599251,17cyclictest0-21swapper/110:34:481
969599251,11cyclictest0-21swapper/112:04:461
969599251,11cyclictest0-21swapper/109:32:551
9695992511,12cyclictest6139-21kworker/u8:008:25:201
970699247,3cyclictest0-21swapper/311:43:333
970699241,13cyclictest0-21swapper/308:08:583
970699241,11cyclictest0-21swapper/312:09:383
970699241,11cyclictest0-21swapper/310:29:263
970699241,11cyclictest0-21swapper/310:12:463
970699240,9cyclictest0-21swapper/310:39:253
970699240,10cyclictest0-21swapper/310:55:293
970199249,4cyclictest0-21swapper/210:35:202
970199240,11cyclictest0-21swapper/209:10:072
969599241,20cyclictest15325-21diskstats07:20:161
969599241,16cyclictest0-21swapper/110:14:491
969599241,12cyclictest0-21swapper/110:23:411
9695992411,11cyclictest30462-21kworker/u8:109:22:241
207672240,2sleep10-21swapper/107:30:271
970699231,10cyclictest0-21swapper/312:28:293
970699230,8cyclictest0-21swapper/310:40:373
970699230,11cyclictest0-21swapper/311:02:373
970699230,10cyclictest0-21swapper/312:18:543
970199230,10cyclictest0-21swapper/211:54:082
969599238,4cyclictest0-21swapper/109:19:241
969599232,13cyclictest87550irq/128-enp1s0-12:26:511
969599231,5cyclictest0-21swapper/111:10:211
969599231,3cyclictest19430-21perf12:25:001
969599231,20cyclictest27943-21perf07:50:001
969599231,11cyclictest0-21swapper/112:39:581
969599231,11cyclictest0-21swapper/111:07:041
969099231,5cyclictest6055-21kernelversion09:25:170
8692230,4sleep0969099cyclictest09:15:120
252162230,2sleep20-21swapper/210:10:132
970699221,9cyclictest0-21swapper/307:22:573
970199221,6cyclictest0-21swapper/210:50:212
969599229,4cyclictest30549-21expr07:55:141
969599229,4cyclictest30549-21expr07:55:141
969599227,3cyclictest0-21swapper/109:57:081
969599226,3cyclictest0-21swapper/112:09:261
969599222,7cyclictest23-21ksoftirqd/110:55:191
969599222,10cyclictest15077-21ntp_states09:45:211
969599221,9cyclictest0-21swapper/111:18:021
969599221,3cyclictest882-21/usr/sbin/munin11:40:201
969599221,3cyclictest3843-21chrt11:47:471
969599221,12cyclictest18121-21df08:40:131
969599221,11cyclictest2613-21expr08:05:121
969599221,11cyclictest0-21swapper/111:21:141
969599221,10cyclictest0-21swapper/110:50:591
969599221,10cyclictest0-21swapper/110:43:231
969099222,4cyclictest0-21swapper/009:52:530
969099221,2cyclictest0-21swapper/011:54:140
969099220,7cyclictest5554-21kworker/u8:010:42:210
970199215,8cyclictest87350irq/126-enp1s0-12:35:152
970199211,4cyclictest0-21swapper/210:34:392
970199211,3cyclictest0-21swapper/210:09:202
969599212,17cyclictest0-21swapper/111:01:291
969599211,4cyclictest0-21swapper/110:05:431
969599211,18cyclictest0-21swapper/111:54:351
969599211,18cyclictest0-21swapper/110:45:151
969599211,18cyclictest0-21swapper/109:11:061
969599211,18cyclictest0-21swapper/107:25:181
969599211,11cyclictest6284-21ntpq09:25:211
969599211,11cyclictest23809-21latency_hist12:35:011
969599211,11cyclictest0-21swapper/112:12:581
969599211,10cyclictest26759-21pmu-power11:25:241
969599211,10cyclictest26759-21pmu-power11:25:231
969599211,10cyclictest0-21swapper/112:19:581
9695992110,3cyclictest27888-21smartctl10:15:151
9695992110,3cyclictest27888-21smartctl10:15:151
969099211,7cyclictest0-21swapper/011:59:020
969099211,5cyclictest18180-21taskset12:20:170
9690992111,5cyclictest4838-21processes10:35:220
969099211,11cyclictest13062-21/usr/sbin/munin12:10:130
970699201,10cyclictest9613-21/usr/sbin/munin09:35:113
970199202,14cyclictest23747-21head08:50:272
969599207,3cyclictest24732-21diskstats08:55:171
969599202,17cyclictest0-21swapper/111:58:101
9695992016,3cyclictest0-21swapper/108:18:221
9695992016,3cyclictest0-21swapper/107:42:321
969599201,10cyclictest4949-21diskstats08:10:151
969599201,10cyclictest0-21swapper/107:10:261
969599200,9cyclictest30462-21kworker/u8:109:37:201
969099209,3cyclictest29402-21/usr/sbin/munin10:20:170
969099209,3cyclictest24384-21/usr/sbin/munin08:55:200
969099207,5cyclictest18245-1kworker/u9:212:08:560
969099207,3cyclictest20547-21tail11:10:250
969099204,8cyclictest8561-21awk09:30:210
969099202,15cyclictest3424-21basename10:35:020
969099201,3cyclictest87350irq/126-enp1s0-08:20:020
969099201,12cyclictest0-21swapper/010:58:080
969099201,10cyclictest0-21swapper/010:29:020
969099201,10cyclictest0-21swapper/010:13:530
970199192,6cyclictest827-21runrttasks12:04:002
970199191,9cyclictest20716-21cpuspeed_turbos10:00:122
970199191,7cyclictest16745-21cat11:05:012
9701991914,3cyclictest16314-21ntp_offset12:15:202
969599192,3cyclictest0-21swapper/108:51:311
969599192,16cyclictest0-21swapper/111:31:211
9695991915,3cyclictest0-21swapper/108:36:021
9695991915,3cyclictest0-21swapper/107:51:131
969599191,10cyclictest20320-21df08:45:141
969599191,10cyclictest1246-21ntpq08:00:211
9695991910,3cyclictest30253-21expr11:35:121
969099199,2cyclictest87550irq/128-enp1s0-11:26:030
969099199,2cyclictest87550irq/128-enp1s0-11:26:030
969099199,2cyclictest28885-21sshd07:50:170
969099195,4cyclictest24920-21latency12:35:190
969099195,4cyclictest23822-21perf12:35:010
969099194,4cyclictest20953-21cat12:25:260
969099193,8cyclictest15305-21hddtemp_smartct11:00:170
969099191,4cyclictest827-21runrttasks09:22:200
9690991914,3cyclictest11866-21awk07:15:010
969099191,16cyclictest4042-21awk11:50:010
969099191,12cyclictest0-21swapper/011:21:430
969099191,10cyclictest4072-21sshd08:06:180
969099191,10cyclictest15760-21sshd09:47:180
970199186,3cyclictest12050irq/123-ahci[0008:25:012
970199186,3cyclictest0-21swapper/207:30:182
970199183,3cyclictest87550irq/128-enp1s0-09:35:142
970199183,12cyclictest18058-21head12:20:162
970199182,6cyclictest22657-21irqstats12:30:172
970199182,6cyclictest13159-21sh08:30:012
970199182,6cyclictest0-21swapper/211:49:512
970199181,7cyclictest19098-21cpu11:10:122
970199181,7cyclictest0-21swapper/212:25:202
970199181,6cyclictest29173-21sh10:20:012
970199181,6cyclictest29173-21sh10:20:012
970199181,2cyclictest0-21swapper/208:32:292
970199181,11cyclictest0-21swapper/209:58:072
970199181,11cyclictest0-21swapper/209:31:112
970199180,10cyclictest0-21swapper/210:23:332
9695991813,4cyclictest0-21swapper/108:30:201
969599181,10cyclictest0-21swapper/109:05:121
969599181,10cyclictest0-21swapper/109:05:121
969599180,16cyclictest0-21swapper/107:15:581
969099189,3cyclictest21733-21sshd08:47:190
969099189,2cyclictest87550irq/128-enp1s0-10:48:410
969099189,2cyclictest87550irq/128-enp1s0-08:20:260
969099188,2cyclictest87550irq/128-enp1s0-11:17:210
969099184,8cyclictest8840-21kworker/u8:110:51:510
969099183,4cyclictest29354-21latency_hist10:20:010
969099183,4cyclictest29354-21latency_hist10:20:010
969099183,2cyclictest87350irq/126-enp1s0-08:32:190
969099182,9cyclictest24464-21sort07:40:170
969099182,5cyclictest15598-21df_inode12:15:130
969099182,4cyclictest14752-21munin-run07:20:020
969099182,14cyclictest17259-21sshd08:36:180
969099181,9cyclictest15652-21hddtemp_smartct07:20:170
969099181,5cyclictest29412-21users11:30:260
969099181,3cyclictest0-21swapper/009:10:200
9690991812,3cyclictest17968-21cstates08:40:130
969099181,11cyclictest0-21swapper/007:55:150
969099181,11cyclictest0-21swapper/007:55:150
970199174,4cyclictest30-21rcuc/208:15:182
970199171,9cyclictest0-21swapper/211:05:132
970199171,9cyclictest0-21swapper/209:00:122
970199171,9cyclictest0-21swapper/208:55:152
970199171,7cyclictest842-21grep09:15:132
970199171,7cyclictest5216-21/usr/sbin/munin09:25:192
970199171,7cyclictest11813-21/usr/sbin/munin09:40:212
970199171,6cyclictest578-21sa111:40:182
970199171,6cyclictest0-21swapper/211:17:282
970199171,5cyclictest13159-21idleruntime10:55:182
970199171,5cyclictest0-21swapper/211:26:512
970199171,5cyclictest0-21swapper/211:26:512
970199171,5cyclictest0-21swapper/207:50:082
970199171,3cyclictest0-21swapper/211:39:352
970199171,3cyclictest0-21swapper/210:46:552
970199171,2cyclictest0-21swapper/210:44:332
970199171,2cyclictest0-21swapper/208:11:572
970199171,2cyclictest0-21swapper/207:10:112
9701991712,3cyclictest31611-21acpi10:25:112
9701991712,3cyclictest21571-21/usr/sbin/munin07:35:212
9701991711,3cyclictest31156-21if_err_enp2s007:55:192
9701991711,3cyclictest31156-21if_err_enp2s007:55:192
970199171,13cyclictest29937-21pmu-power09:05:212
970199171,13cyclictest29937-21pmu-power09:05:212
970199171,13cyclictest17124-21grep08:35:262
970199171,10cyclictest0-21swapper/211:56:162
969099175,3cyclictest25962-21/usr/sbin/munin07:45:140
969099174,3cyclictest20763-21cpuspeed_turbos10:00:120
969099174,3cyclictest1755-21head08:00:250
969099172,12cyclictest18412-21/usr/sbin/munin09:55:150
969099171,4cyclictest6873-21kworker/0:009:35:200
969099171,2cyclictest0-21swapper/012:00:230
969099171,2cyclictest0-21swapper/010:05:160
9690991712,3cyclictest130902sleep008:29:140
969099171,15cyclictest0-21swapper/009:02:570
969099171,14cyclictest0-21swapper/007:26:570
970199169,4cyclictest0-21swapper/208:05:122
970199161,7cyclictest0-21swapper/209:45:182
970199161,6cyclictest0-21swapper/209:21:332
970199161,6cyclictest0-21swapper/207:25:172
970199161,2cyclictest0-21swapper/207:23:092
970199161,14cyclictest0-21swapper/208:42:422
970199161,14cyclictest0-21swapper/207:18:592
9701991611,3cyclictest692-21grep08:00:152
9701991611,3cyclictest24697-21sort07:40:202
970199161,12cyclictest28428-21grep11:30:162
9701991610,3cyclictest20163-21cat08:45:122
969099161,8cyclictest808-21snmpd07:38:020
969099161,3cyclictest0-21swapper/008:10:240
969099158,4cyclictest29444-21grep09:05:160
969099158,4cyclictest29444-21grep09:05:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional