You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-27 - 17:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Mon Apr 27, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
127052238164,24sleep20-21swapper/207:07:142
127902203165,26sleep00-21swapper/007:08:210
126032202165,25sleep30-21swapper/307:05:543
109562201164,24sleep10-21swapper/107:05:041
340321770,4sleep21301199cyclictest09:10:262
13011991131,110cyclictest0-21swapper/208:55:592
102042680,6sleep01299799cyclictest10:40:220
13011994830,4cyclictest32053-21memory11:30:192
1299799369,13cyclictest0-21swapper/009:45:010
1299799369,13cyclictest0-21swapper/009:45:010
1299799368,13cyclictest3925-21idleruntime-cro09:15:010
1299799335,13cyclictest20879-21munin-run08:40:010
1299799327,12cyclictest8925-21munin-run10:40:010
1299799321,23cyclictest87350irq/126-enp1s0-09:20:230
1301199311,21cyclictest0-21swapper/209:00:112
1301199311,20cyclictest0-21swapper/208:08:192
1301199311,10cyclictest1715-21diskmemload10:35:032
13016993015,9cyclictest0-21swapper/309:42:043
13016993015,9cyclictest0-21swapper/309:42:043
13011993026,3cyclictest0-21swapper/209:54:102
1301199301,20cyclictest0-21swapper/210:44:072
1301199301,12cyclictest0-21swapper/210:10:512
1301199300,13cyclictest0-21swapper/209:33:472
1301699291,20cyclictest0-21swapper/311:43:293
1301199291,23cyclictest0-21swapper/209:23:032
1301199291,14cyclictest0-21swapper/211:43:232
1301199291,11cyclictest1010-21hddtemp_smartct10:20:152
1301199291,11cyclictest0-21swapper/210:01:032
1299799291,13cyclictest19932-21munin-run11:05:010
1301699283,18cyclictest0-21swapper/310:54:453
1301699283,18cyclictest0-21swapper/310:54:453
1301699281,19cyclictest0-21swapper/311:18:093
1301199281,13cyclictest0-21swapper/212:29:152
1301199281,12cyclictest0-21swapper/212:20:072
1301199281,11cyclictest0-21swapper/210:09:432
1299799285,11cyclictest7359-21perf11:50:010
1301699272,17cyclictest0-21swapper/310:42:543
13016992715,3cyclictest0-21swapper/311:25:193
1301699271,18cyclictest0-21swapper/312:37:273
1301699271,18cyclictest0-21swapper/311:57:123
1301699271,18cyclictest0-21swapper/311:11:383
1301699271,18cyclictest0-21swapper/310:58:553
1301199279,9cyclictest0-21swapper/207:40:072
1301199279,3cyclictest0-21swapper/211:00:312
1301199277,9cyclictest0-21swapper/209:06:592
1301199277,9cyclictest0-21swapper/208:38:392
13011992713,3cyclictest0-21swapper/211:51:192
1301199271,12cyclictest0-21swapper/212:38:472
1301199271,12cyclictest0-21swapper/212:11:072
1301199271,12cyclictest0-21swapper/210:52:272
1301199271,12cyclictest0-21swapper/210:52:272
1301199271,10cyclictest1715-21diskmemload11:06:152
1301199271,10cyclictest0-21swapper/211:19:112
1301199271,10cyclictest0-21swapper/211:13:032
13011992710,4cyclictest0-21swapper/209:15:142
1300699271,19cyclictest0-21swapper/112:21:431
1299799271,12cyclictest31269-21perf07:50:010
13016992620,3cyclictest6009-21mailstats10:30:223
13016992620,3cyclictest30548-21cpu10:15:123
1301699261,18cyclictest0-21swapper/311:30:493
1301699261,17cyclictest0-21swapper/311:05:393
1301699261,16cyclictest0-21swapper/309:28:093
1301699260,17cyclictest0-21swapper/309:10:043
1301199268,9cyclictest0-21swapper/208:45:072
1301199266,3cyclictest0-21swapper/212:18:272
1301199261,9cyclictest0-21swapper/211:20:312
13011992613,3cyclictest1715-21diskmemload12:06:432
13011992613,3cyclictest1715-21diskmemload12:06:432
13011992612,3cyclictest0-21swapper/209:42:152
13011992612,3cyclictest0-21swapper/209:42:152
1301199261,17cyclictest0-21swapper/210:17:352
1301199261,13cyclictest0-21swapper/211:29:192
1301199261,12cyclictest0-21swapper/210:46:232
1301199261,11cyclictest0-21swapper/210:31:152
1301199261,11cyclictest0-21swapper/210:26:472
13011992610,3cyclictest0-21swapper/209:49:072
1301699254,14cyclictest5854-21kworker/u8:009:50:453
1301699251,18cyclictest87350irq/126-enp1s0-08:04:203
1301699251,18cyclictest0-21swapper/311:21:473
1301699251,18cyclictest0-21swapper/311:00:523
1301699251,18cyclictest0-21swapper/310:01:523
1301699251,17cyclictest11380-21kworker/u8:112:02:213
1301699251,17cyclictest0-21swapper/309:59:093
1301699251,17cyclictest0-21swapper/309:31:473
1301699251,16cyclictest87350irq/126-enp1s0-07:25:203
1301699251,16cyclictest0-21swapper/312:12:233
1301699250,17cyclictest0-21swapper/311:52:333
1301699250,17cyclictest0-21swapper/310:21:453
1301199258,3cyclictest0-21swapper/207:51:552
1301199258,3cyclictest0-21swapper/207:39:112
13011992517,3cyclictest0-21swapper/208:54:112
13011992514,3cyclictest0-21swapper/212:30:512
1301199251,13cyclictest0-21swapper/207:29:552
1301199251,11cyclictest0-21swapper/211:36:112
13006992514,3cyclictest25502-21df_inode12:30:131
1300699251,16cyclictest17782-21latency_hist11:00:011
1299799257,15cyclictest25918-21perf10:05:010
1299799251,12cyclictest1727-21awk09:10:010
1301699241,17cyclictest0-21swapper/310:45:023
1301699241,17cyclictest0-21swapper/309:16:503
1301699241,16cyclictest0-21swapper/310:13:453
1301699241,16cyclictest0-21swapper/309:49:433
1301699241,16cyclictest0-21swapper/309:20:303
1301699240,22cyclictest0-21swapper/312:19:013
1301199248,3cyclictest0-21swapper/207:48:272
1301199246,9cyclictest0-21swapper/212:00:032
13011992416,3cyclictest0-21swapper/208:23:112
13011992415,3cyclictest0-21swapper/207:56:232
1301199241,15cyclictest0-21swapper/208:30:192
1301199241,15cyclictest0-21swapper/208:30:192
13011992411,3cyclictest0-21swapper/208:00:272
1301199241,11cyclictest0-21swapper/211:55:272
1301199241,11cyclictest0-21swapper/211:45:552
1301199241,11cyclictest0-21swapper/209:36:032
1301199240,11cyclictest0-21swapper/207:30:512
1300699249,13cyclictest4361-21kworker/u8:211:02:141
1300699243,9cyclictest0-21swapper/110:50:201
1300699243,9cyclictest0-21swapper/110:50:201
1300699241,4cyclictest0-21swapper/112:05:231
1300699241,4cyclictest0-21swapper/112:05:231
1300699241,17cyclictest0-21swapper/111:24:301
1300699241,10cyclictest0-21swapper/112:10:561
1300699240,18cyclictest87550irq/128-enp1s0-09:20:311
1299799242,11cyclictest955-21/usr/sbin/munin11:35:150
1299799241,11cyclictest14179-21/usr/sbin/munin12:05:170
1299799241,11cyclictest14179-21/usr/sbin/munin12:05:170
1299799240,22cyclictest0-21swapper/009:45:260
1301699236,9cyclictest0-21swapper/309:35:033
13016992316,4cyclictest32471-21head07:50:193
13016992316,4cyclictest0-21swapper/310:29:483
1301699231,17cyclictest0-21swapper/312:34:153
1301699231,17cyclictest0-21swapper/312:27:483
1301699231,16cyclictest0-21swapper/312:21:363
1301699231,16cyclictest0-21swapper/312:06:293
1301699231,16cyclictest0-21swapper/312:06:283
1301699231,15cyclictest0-21swapper/310:37:283
1301199238,8cyclictest0-21swapper/209:27:312
1301199238,3cyclictest0-21swapper/208:17:592
1301199238,3cyclictest0-21swapper/207:22:352
1301199237,3cyclictest0-21swapper/208:27:472
1301199236,3cyclictest0-21swapper/207:16:402
1301199236,3cyclictest0-21swapper/207:16:392
13011992315,3cyclictest0-21swapper/209:57:112
1301199230,12cyclictest0-21swapper/207:11:592
1300699232,18cyclictest22300-21perl09:55:141
13006992317,3cyclictest18986-21taskset09:46:091
13006992315,3cyclictest0-21swapper/110:41:091
1300699231,3cyclictest4429-21memory08:00:171
1300699231,3cyclictest27424-21cpuspeed_turbos12:35:111
1300699231,14cyclictest0-21swapper/111:50:111
1300699231,10cyclictest0-21swapper/108:51:531
1300699230,6cyclictest0-21swapper/109:25:161
1299799232,10cyclictest20153-21/usr/sbin/munin11:05:180
1299799231,6cyclictest0-21swapper/012:16:280
1299799231,20cyclictest579-21systemd-journal08:29:430
1299799231,11cyclictest25161-21/usr/sbin/munin12:30:260
315802221,3sleep131810-21hddtemp_smartct11:30:161
1301699227,9cyclictest0-21swapper/311:46:113
1301699227,9cyclictest0-21swapper/307:48:433
1301699221,5cyclictest0-21swapper/308:52:203
1301699221,15cyclictest0-21swapper/311:37:553
1301699221,15cyclictest0-21swapper/308:39:433
1301699221,13cyclictest87350irq/126-enp1s0-08:45:173
1301199227,1cyclictest0-21swapper/210:59:352
1301199226,3cyclictest0-21swapper/208:43:482
1300699229,3cyclictest0-21swapper/110:16:291
1300699222,11cyclictest21453-21diskstats08:40:151
13006992217,3cyclictest28722-21nfs4_client08:55:211
1300699221,3cyclictest6226-21grep08:05:141
1300699221,3cyclictest31262-21perf07:50:011
1300699221,15cyclictest0-21swapper/109:17:311
1300699221,15cyclictest0-21swapper/107:10:161
1300699221,11cyclictest24842-21missed_timers10:00:191
1300699221,11cyclictest0-21swapper/110:28:151
1299799229,2cyclictest26130-21/usr/sbin/munin10:05:180
1299799226,4cyclictest13057-21sshd10:46:430
1299799222,3cyclictest12050irq/123-ahci[0011:35:010
1299799220,20cyclictest0-21swapper/009:15:130
1299799220,20cyclictest0-21swapper/008:40:190
1301699213,9cyclictest30838-21wc09:00:193
1301699211,14cyclictest0-21swapper/308:28:243
1301199219,3cyclictest0-21swapper/208:11:272
1300699219,11cyclictest26733-21kworker/u8:111:46:241
1300699217,4cyclictest0-21swapper/110:49:141
1300699216,3cyclictest0-21swapper/107:52:191
1300699211,15cyclictest0-21swapper/111:15:471
1300699211,14cyclictest0-21swapper/111:26:221
1300699211,14cyclictest0-21swapper/107:42:401
1300699211,11cyclictest15284-21grep09:40:131
1300699211,11cyclictest15284-21grep09:40:131
1300699211,11cyclictest0-21swapper/109:11:541
1300699211,10cyclictest1715-21diskmemload11:10:011
1300699211,10cyclictest0-21swapper/112:15:421
1300699211,10cyclictest0-21swapper/110:34:281
1300699211,10cyclictest0-21swapper/110:24:141
1300699211,10cyclictest0-21swapper/109:38:471
1300699210,15cyclictest0-21swapper/107:38:391
1300699210,13cyclictest0-21swapper/112:00:121
1299799211,19cyclictest0-21swapper/008:48:550
1299799210,20cyclictest0-21swapper/011:55:180
178642201,2sleep10-21swapper/107:17:331
178642201,2sleep10-21swapper/107:17:331
1301699201,7cyclictest8255-21df08:10:133
1301699201,7cyclictest14412-21cat08:25:013
1301699201,6cyclictest26851-21latency_hist07:40:023
1301699201,6cyclictest19673-21wc07:20:243
1301699201,5cyclictest32209-21grep09:05:133
1300699209,3cyclictest15779-21smart_sda08:25:251
1300699202,10cyclictest0-21swapper/110:10:201
13006992017,2cyclictest0-21swapper/110:36:231
13006992013,3cyclictest0-21swapper/111:36:171
13006992012,6cyclictest9922-21kworker/u8:108:21:481
1300699201,14cyclictest0-21swapper/112:26:001
1300699201,14cyclictest0-21swapper/111:57:131
1300699201,14cyclictest0-21swapper/107:58:351
1300699201,14cyclictest0-21swapper/107:31:161
1300699201,14cyclictest0-21swapper/107:23:291
1300699201,11cyclictest31868-21latency_hist09:05:021
1300699201,10cyclictest17783-21ntp_states08:30:221
1300699201,10cyclictest17783-21ntp_states08:30:221
1300699200,18cyclictest0-21swapper/108:35:131
1300699200,18cyclictest0-21swapper/108:15:211
1300699200,11cyclictest0-21swapper/111:10:231
1299799206,3cyclictest87550irq/128-enp1s0-12:20:250
1299799203,8cyclictest731650kworker/u8:211:50:480
1299799202,5cyclictest24894-21diskstats11:15:130
1299799201,7cyclictest1276-21missed_timers10:20:180
1299799201,6cyclictest25701-21irqstats07:35:180
1299799201,15cyclictest14615-21ntp_states10:50:220
1299799201,15cyclictest14615-21ntp_states10:50:210
1299799200,7cyclictest10282-21kworker/u8:209:35:320
1299799200,18cyclictest0-21swapper/009:30:180
1299799200,10cyclictest0-21swapper/010:15:140
1299799200,10cyclictest0-21swapper/009:00:190
1301699199,8cyclictest18084-21awk07:20:013
1301699199,8cyclictest18084-21awk07:20:013
1301699196,3cyclictest87650irq/129-enp1s0-07:55:213
1301699194,4cyclictest1715-21diskmemload10:05:223
1301699191,6cyclictest16878-21cstates08:30:133
1301699191,6cyclictest16878-21cstates08:30:133
1301699191,5cyclictest0-21swapper/309:00:023
1301699191,10cyclictest22687-21/usr/sbin/munin07:30:143
1300699198,3cyclictest26465-21df10:05:141
13006991915,3cyclictest0-21swapper/109:51:461
13006991915,3cyclictest0-21swapper/109:07:471
1300699191,11cyclictest11904-21processes09:30:221
1299799195,3cyclictest87350irq/126-enp1s0-11:40:150
1299799195,3cyclictest19525-21/usr/sbin/munin09:50:150
1299799191,5cyclictest9061-21head08:10:220
1299799191,4cyclictest22767-21grep11:10:140
1299799191,4cyclictest12495-21grep12:00:150
12997991914,3cyclictest29311-21fschecks_count11:25:150
12997991913,3cyclictest23633-21gltestperf12:25:140
1299799191,16cyclictest87350irq/126-enp1s0-08:50:210
1299799191,16cyclictest87350irq/126-enp1s0-08:00:110
1299799191,16cyclictest87350irq/126-enp1s0-07:55:170
1299799191,16cyclictest87350irq/126-enp1s0-07:11:430
1299799191,15cyclictest8532-21/usr/sbin/munin09:25:160
1299799190,18cyclictest0-21swapper/008:33:520
1299799190,18cyclictest0-21swapper/008:33:520
1301699184,4cyclictest13206-21/usr/sbin/munin07:10:133
1301699184,3cyclictest165050irq/12-ttyS008:40:203
1301699181,6cyclictest10753-21gltestperf08:15:163
1301699181,5cyclictest28344-21smart_sda07:40:233
1301699181,5cyclictest0-21swapper/308:05:223
13006991814,3cyclictest0-21swapper/108:13:171
1300699181,16cyclictest0-21swapper/108:45:081
13006991810,3cyclictest0-21swapper/111:42:441
1299799185,2cyclictest4756-21/usr/sbin/munin10:30:130
1299799184,3cyclictest87550irq/128-enp1s0-08:20:230
1299799184,3cyclictest87550irq/128-enp1s0-07:40:140
1299799183,3cyclictest29662-21latency_hist09:00:020
1299799182,4cyclictest21721-21/usr/sbin/munin09:55:230
1299799182,13cyclictest6681-21sed08:05:180
1299799181,7cyclictest17918-21unixbench_singl12:10:250
1299799181,5cyclictest28321-21/usr/sbin/munin10:10:220
1299799181,4cyclictest0-21swapper/010:26:020
12997991813,3cyclictest12087-21munin-run08:20:010
1299799181,14cyclictest22814-21tr07:30:110
12997991811,4cyclictest21471-21/usr/sbin/munin07:25:200
1299799180,9cyclictest0-21swapper/007:20:140
1299799180,17cyclictest0-21swapper/007:50:260
1300699171,15cyclictest0-21swapper/107:29:501
1299799174,3cyclictest87550irq/128-enp1s0-07:15:150
1299799174,3cyclictest87550irq/128-enp1s0-07:15:150
1299799172,3cyclictest0-21swapper/010:56:520
1299799171,9cyclictest0-21swapper/012:35:300
12997991711,3cyclictest26745-21/usr/sbin/munin11:20:170
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional