You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-15 - 09:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Mon Jun 15, 2026 00:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
323572236162,24sleep20-21swapper/219:06:062
325852231161,57sleep10-21swapper/119:09:021
326092221190,21sleep00-21swapper/019:09:220
307012203166,24sleep30-21swapper/319:05:103
1069921780,4sleep13274799cyclictest19:30:171
1130021700,3sleep2311ktimersoftd/222:00:112
3150721380,5sleep13274799cyclictest00:00:011
109442650,3sleep339-21rcuc/320:45:113
291072640,4sleep230-21rcuc/222:40:122
29422610,17sleep00-21swapper/000:05:260
221212490,4sleep011-21rcuc/022:25:010
221212490,4sleep011-21rcuc/022:25:010
191632420,3sleep019162-21gltestperf23:30:150
191632420,3sleep019162-21gltestperf23:30:150
281392350,1sleep10-21swapper/123:50:181
3274799321,13cyclictest0-21swapper/119:42:011
3273899321,24cyclictest0-21swapper/000:17:400
3274799311,8cyclictest0-21swapper/122:16:491
3274799311,8cyclictest0-21swapper/121:12:291
3274799311,8cyclictest0-21swapper/121:12:291
3274799311,12cyclictest0-21swapper/121:56:411
3273899311,23cyclictest0-21swapper/023:26:030
3274799301,21cyclictest0-21swapper/121:18:291
3274799301,21cyclictest0-21swapper/100:03:551
3274799301,14cyclictest0-21swapper/123:22:331
3274799301,14cyclictest0-21swapper/100:26:291
3274799301,12cyclictest0-21swapper/121:35:091
3274799301,12cyclictest0-21swapper/100:35:091
3274799301,11cyclictest0-21swapper/123:34:091
3274799301,11cyclictest0-21swapper/123:34:091
3274799301,11cyclictest0-21swapper/121:51:411
3274799301,11cyclictest0-21swapper/121:47:531
3274799301,10cyclictest0-21swapper/120:34:211
3273899301,14cyclictest16441-21munin-run19:45:010
3274799291,12cyclictest0-21swapper/122:36:491
3274799291,12cyclictest0-21swapper/100:11:331
3274799291,11cyclictest0-21swapper/123:16:251
3274799291,11cyclictest0-21swapper/122:12:091
3273899294,12cyclictest8883-21munin-run21:55:010
32738992910,16cyclictest7654-21munin-run19:25:010
3273899290,22cyclictest87350irq/126-enp1s0-21:10:030
3273899290,22cyclictest87350irq/126-enp1s0-21:10:030
3274799281,9cyclictest0-21swapper/121:09:571
3274799281,8cyclictest0-21swapper/120:42:371
3274799281,12cyclictest0-21swapper/122:00:371
3274799281,11cyclictest0-21swapper/123:43:251
3274799281,11cyclictest0-21swapper/123:08:451
3274799281,11cyclictest0-21swapper/122:33:211
3274799281,10cyclictest0-21swapper/123:13:251
3274799281,10cyclictest0-21swapper/100:18:451
3274799280,8cyclictest0-21swapper/123:35:491
3273899288,13cyclictest87350irq/126-enp1s0-21:49:310
3275499272,11cyclictest0-21swapper/223:45:492
3274799271,12cyclictest0-21swapper/121:29:491
3274799271,11cyclictest0-21swapper/123:02:251
3274799271,10cyclictest0-21swapper/123:46:291
3275999264,8cyclictest4200-40dnf21:41:373
3275999261,4cyclictest0-21swapper/321:51:523
3274799266,10cyclictest3805-21expr19:15:131
3274799261,12cyclictest0-21swapper/122:53:571
3274799261,11cyclictest6966-21hddtemp_smartct20:35:171
3274799261,11cyclictest0-21swapper/122:21:371
3274799261,11cyclictest0-21swapper/122:21:371
3274799261,11cyclictest0-21swapper/120:55:291
3274799261,11cyclictest0-21swapper/120:10:371
3274799261,10cyclictest0-21swapper/122:25:411
3274799261,10cyclictest0-21swapper/122:09:491
3274799261,10cyclictest0-21swapper/120:01:571
3274799261,10cyclictest0-21swapper/120:01:571
3274799261,10cyclictest0-21swapper/100:31:011
3275499251,5cyclictest0-21swapper/222:50:022
3274799258,2cyclictest27236-21kworker/u8:120:17:471
3274799251,9cyclictest0-21swapper/121:30:331
3274799251,12cyclictest0-21swapper/122:59:091
3274799251,11cyclictest0-21swapper/122:41:331
3274799251,10cyclictest0-21swapper/122:49:411
3274799251,10cyclictest0-21swapper/100:20:571
3273899252,11cyclictest9450-21diskstats21:55:140
3273899251,12cyclictest29292-21perf23:55:010
3275999248,3cyclictest0-21swapper/321:01:193
3275999242,11cyclictest16315-21/usr/sbin/munin23:25:143
3275999241,5cyclictest0-21swapper/323:13:253
3275499246,9cyclictest0-21swapper/200:25:222
3274799244,3cyclictest0-21swapper/100:08:171
3274799241,16cyclictest0-21swapper/119:56:331
3274799241,15cyclictest0-21swapper/121:42:531
3274799241,12cyclictest0-21swapper/120:54:491
3274799241,11cyclictest18338-21ntp_states21:00:211
3273899241,17cyclictest0-21swapper/022:43:100
3273899241,16cyclictest0-21swapper/022:39:410
3273899241,15cyclictest0-21swapper/021:26:160
3273899241,11cyclictest16690-21strings22:10:200
3273899241,11cyclictest14317-21irqstats22:05:190
3273899241,11cyclictest14068-21sed00:30:250
3275999233,11cyclictest22908-21/usr/sbin/munin23:40:213
3275999231,11cyclictest24493-21/usr/sbin/munin22:30:253
3275499236,4cyclictest87650irq/129-enp1s0-21:55:212
3275499232,5cyclictest14856-21munin-run20:55:012
3274799238,3cyclictest0-21swapper/120:25:531
3274799236,10cyclictest0-21swapper/121:20:091
3274799231,6cyclictest0-21swapper/119:36:171
3274799231,18cyclictest16933-21gltestperf23:25:151
3274799231,11cyclictest0-21swapper/119:51:131
3273899231,16cyclictest0-21swapper/023:38:110
3273899231,15cyclictest0-21swapper/023:45:160
3273899231,11cyclictest0-21swapper/023:00:160
3273899231,10cyclictest10363-21/usr/sbin/munin00:25:150
3273899230,16cyclictest0-21swapper/023:44:370
3273899230,14cyclictest87350irq/126-enp1s0-22:45:070
147622230,2sleep00-21swapper/000:35:120
61002220,5sleep339-21rcuc/300:15:123
3275999221,13cyclictest0-21swapper/320:25:263
3274799226,3cyclictest0-21swapper/119:45:331
3274799221,9cyclictest0-21swapper/119:12:051
3274799221,10cyclictest0-21swapper/120:22:251
3273899228,3cyclictest0-21swapper/023:55:280
3273899222,8cyclictest9600-21timerwakeupswit00:20:270
3273899222,11cyclictest0-21swapper/021:22:010
3273899221,6cyclictest31147-21smartctl21:30:160
3273899221,12cyclictest0-21swapper/023:06:120
3273899221,12cyclictest0-21swapper/021:35:550
3273899221,10cyclictest4064-21head21:40:260
3273899221,10cyclictest20990-21grep21:05:270
3273899221,10cyclictest0-21swapper/022:31:590
3273899220,6cyclictest26346-21kworker/u8:222:58:110
13802221,2sleep00-21swapper/019:10:180
128842220,5sleep011-21rcuc/022:00:250
3275999212,10cyclictest10609-21unixbench_multi21:55:253
32759992114,4cyclictest20623-21grep22:20:153
32759992114,4cyclictest20623-21grep22:20:153
3275999211,12cyclictest0-21swapper/322:45:113
3275999211,10cyclictest5709-21fschecks_count23:00:143
32759992110,3cyclictest0-21swapper/321:37:353
3275499216,9cyclictest0-21swapper/222:10:222
3275499215,4cyclictest21312-21grep23:35:152
3275499215,4cyclictest0-21swapper/221:05:122
3275499214,4cyclictest26621-21fschecks_time21:20:132
3275499214,3cyclictest87650irq/129-enp1s0-00:15:182
3275499211,8cyclictest0-21swapper/221:25:182
3275499211,7cyclictest0-21swapper/222:19:482
3274799217,3cyclictest0-21swapper/119:22:131
3274799216,3cyclictest0-21swapper/120:45:571
3274799212,10cyclictest9130-21sensors19:25:211
3273899218,3cyclictest0-21swapper/020:35:150
3273899217,3cyclictest0-21swapper/020:40:260
3273899216,6cyclictest0-21swapper/000:00:090
3273899212,11cyclictest32051-21/usr/sbin/munin20:20:150
3273899212,10cyclictest29002-21kworker/0:100:10:130
3273899211,9cyclictest0-21swapper/023:10:230
3273899211,9cyclictest0-21swapper/020:29:350
3273899211,14cyclictest0-21swapper/023:21:280
3273899211,11cyclictest0-21swapper/022:15:040
3273899211,11cyclictest0-21swapper/021:17:080
3275999209,3cyclictest23877-21/usr/sbin/munin21:15:253
3275999206,3cyclictest0-21swapper/320:58:193
32759992014,3cyclictest6665-21needreboot19:20:193
3275999201,12cyclictest0-21swapper/322:59:193
3275999201,11cyclictest11844-21NetworkManager19:32:363
3275999201,11cyclictest0-21swapper/323:05:243
3275999201,10cyclictest13958-21sshd19:36:353
3275999201,10cyclictest13725-21kernelversion20:50:183
3275999201,10cyclictest12305-21fschecks_count23:15:143
3275499206,3cyclictest87650irq/129-enp1s0-23:40:152
3275499206,3cyclictest87650irq/129-enp1s0-22:35:252
3275499206,3cyclictest87650irq/129-enp1s0-21:40:182
3275499205,8cyclictest87350irq/126-enp1s0-21:15:212
3275499205,5cyclictest2467-21latency00:05:192
3275499205,4cyclictest909-21sssd_nss20:55:192
3275499205,4cyclictest21450-21diskmemload00:21:112
3275499204,8cyclictest87550irq/128-enp1s0-23:16:352
3275499202,15cyclictest24751-21unixbench_multi20:00:262
3275499202,15cyclictest24751-21unixbench_multi20:00:262
3275499201,7cyclictest0-21swapper/223:20:592
3275499201,6cyclictest0-21swapper/221:32:582
32754992014,3cyclictest28103-21fschecks_count20:10:142
32754992013,4cyclictest11052-21needreboot19:30:202
3275499201,14cyclictest0-21swapper/219:18:512
3274799205,4cyclictest26638-21ps20:05:251
3273899205,3cyclictest11263-21gltestperf20:45:140
3273899201,7cyclictest808-21snmpd20:19:340
3273899201,7cyclictest13897-21munin-run23:20:010
3273899201,16cyclictest0-21swapper/022:25:070
3273899201,10cyclictest0-21swapper/021:02:130
312362201,2sleep20-21swapper/223:57:102
3275999199,2cyclictest0-21swapper/322:15:133
3275999194,9cyclictest0-21swapper/320:10:123
3275999194,8cyclictest0-21swapper/323:37:323
3275999191,9cyclictest23463-21ps22:25:223
3275999191,7cyclictest0-21swapper/300:26:203
3275999191,6cyclictest0-21swapper/321:22:053
3275999191,5cyclictest21794-21tr21:10:123
3275999191,5cyclictest21794-21tr21:10:123
3275999191,5cyclictest0-21swapper/300:24:143
3275999191,4cyclictest1595-21hddtemp_smartct22:50:163
3275999191,4cyclictest0-21swapper/300:10:143
32759991913,3cyclictest17016-21entropy19:45:153
3275999191,15cyclictest30673-21ps23:55:233
3275999191,12cyclictest0-21swapper/323:33:573
3275999191,12cyclictest0-21swapper/323:33:573
3275999191,12cyclictest0-21swapper/320:05:173
3275999191,10cyclictest15346-21gzip23:20:233
3275999191,10cyclictest0-21swapper/322:10:183
3275499195,4cyclictest9408-21kworker/u8:023:25:262
3275499195,3cyclictest87650irq/129-enp1s0-22:55:132
3275499195,3cyclictest6307-21cpu20:35:132
3275499194,3cyclictest6280-21grep19:20:152
3275499193,9cyclictest7530-21/usr/sbin/munin23:05:162
3275499193,3cyclictest87650irq/129-enp1s0-19:35:282
3275499191,9cyclictest18269-21cut21:00:202
3275499191,7cyclictest16777-21latency_hist00:40:012
3275499191,7cyclictest0-21swapper/200:30:132
3275499191,6cyclictest9341-21unixbench-2d19:25:232
3275499191,6cyclictest332-21if_enp2s020:20:172
3275499191,6cyclictest27527-21expr23:50:132
3275499191,6cyclictest0-21swapper/222:46:232
3275499191,6cyclictest0-21swapper/221:46:022
3275499191,5cyclictest5670-21diskstats23:00:132
3275499191,5cyclictest22848-21perl22:25:152
3275499191,5cyclictest22137-21fschecks_count21:10:152
3275499191,5cyclictest22137-21fschecks_count21:10:152
32754991914,3cyclictest20483-21ls23:35:012
32754991914,3cyclictest20483-21ls23:35:012
3275499191,3cyclictest0-21swapper/219:50:152
3273899195,3cyclictest15079-21/usr/sbin/munin20:55:200
3273899191,9cyclictest0-21swapper/022:54:310
3273899191,8cyclictest0-21swapper/019:50:250
3273899191,2cyclictest0-21swapper/020:10:160
3273899191,16cyclictest0-21swapper/019:55:240
3273899191,16cyclictest0-21swapper/019:30:210
3273899191,15cyclictest24811-21grep20:00:260
3273899191,15cyclictest24811-21grep20:00:260
3273899190,10cyclictest0-21swapper/019:25:190
315922190,4sleep339-21rcuc/321:30:223
131142191,3sleep3808-21snmpd22:03:023
3275999187,3cyclictest87650irq/129-enp1s0-21:45:153
3275999183,4cyclictest14173-21gltestperf22:05:173
3275999181,7cyclictest9459-21users19:25:233
3275999181,7cyclictest0-21swapper/322:37:363
3275999181,7cyclictest0-21swapper/319:55:163
3275999181,7cyclictest0-21swapper/319:41:353
3275999181,6cyclictest0-21swapper/300:05:443
3275999181,5cyclictest28577-21expr21:25:143
3275999181,5cyclictest0-21swapper/300:35:033
32759991812,3cyclictest5325-21find20:30:243
32759991811,4cyclictest9037-21perl20:40:153
3275999181,10cyclictest0-21swapper/300:34:363
3275499185,3cyclictest87650irq/129-enp1s0-22:30:172
3275499185,3cyclictest87650irq/129-enp1s0-19:55:122
3275499184,3cyclictest26459-21needreboot20:05:212
3275499184,3cyclictest20210-21cpuspeed_turbos22:20:122
3275499184,3cyclictest20210-21cpuspeed_turbos22:20:122
3275499183,9cyclictest0-21swapper/221:51:352
3275499183,4cyclictest31851-21tr00:00:142
3275499182,5cyclictest10680-21/usr/sbin/munin20:45:212
3275499181,7cyclictest9474-21needreboot20:40:192
3275499181,7cyclictest0-21swapper/219:10:182
3275499181,6cyclictest0-21swapper/223:10:242
3275499181,5cyclictest4092-21/usr/sbin/munin20:30:182
3275499181,5cyclictest2230-21df20:25:132
3275499181,5cyclictest13526-21grep22:05:132
32754991813,3cyclictest3802-21cpu00:10:132
3275499181,2cyclictest0-21swapper/221:35:132
3273899183,3cyclictest13531-21smart_sda19:35:240
3273899181,5cyclictest4427-21df20:30:120
3273899181,5cyclictest17468-21interrupts19:45:180
3273899181,5cyclictest12879-21/usr/sbin/munin20:50:140
3273899181,3cyclictest4627-21pmu-power19:15:220
3275999179,5cyclictest32693-21gltestperf20:20:163
3275999173,6cyclictest41-21ksoftirqd/319:10:253
3275999171,9cyclictest0-21swapper/320:15:163
3275999171,7cyclictest0-21swapper/323:45:173
3275999171,6cyclictest0-21swapper/319:50:263
3275999171,3cyclictest28997-21sshd23:51:353
32759991713,3cyclictest0-21swapper/321:08:043
3275999171,15cyclictest0-21swapper/322:40:033
32759991711,3cyclictest827-21runrttasks20:37:183
3275999171,11cyclictest0-21swapper/300:01:233
3275499175,2cyclictest87550irq/128-enp1s0-19:45:262
3275499174,3cyclictest87650irq/129-enp1s0-20:15:142
3275499174,3cyclictest87650irq/129-enp1s0-19:40:242
3273899171,15cyclictest0-21swapper/020:06:110
3275999169,4cyclictest23395-21cpuspeed_turbos20:00:133
3275999169,4cyclictest23395-21cpuspeed_turbos20:00:133
3275999161,6cyclictest4394-21memory19:15:183
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional