You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-20 - 10:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Sat Jun 20, 2026 00:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
103822204168,24sleep20-21swapper/219:05:382
105842203165,25sleep10-21swapper/119:08:131
105392203165,25sleep30-21swapper/319:07:403
106502201164,25sleep00-21swapper/019:09:060
1764821590,2sleep31082099cyclictest21:50:133
10815991060,2cyclictest0-21swapper/200:11:302
108159910191,9cyclictest0-21swapper/219:51:262
10815997868,9cyclictest0-21swapper/220:19:412
10815997364,8cyclictest0-21swapper/222:55:012
171272580,4sleep339-21rcuc/321:50:013
10815995450,3cyclictest0-21swapper/221:33:252
244972320,5sleep21081599cyclictest00:30:252
1080999323,16cyclictest0-21swapper/122:55:071
10809993210,5cyclictest0-21swapper/123:48:311
1080399323,14cyclictest13964-21munin-run00:10:010
1080999310,7cyclictest0-21swapper/100:00:011
177392300,3sleep10-21swapper/121:50:151
1080999300,19cyclictest0-21swapper/123:30:011
1080399302,13cyclictest8305-21munin-run21:30:020
1082099296,17cyclictest0-21swapper/322:59:413
1080999291,18cyclictest0-21swapper/100:08:441
1080999290,16cyclictest0-21swapper/123:31:581
1080999281,17cyclictest0-21swapper/121:36:141
1080999280,21cyclictest18283-21kworker/u8:000:28:261
10803992810,15cyclictest12102-21perf20:25:010
10815992724,2cyclictest0-21swapper/219:13:162
1080999278,9cyclictest0-21swapper/100:20:501
1080999278,9cyclictest0-21swapper/100:20:501
1080999273,15cyclictest0-21swapper/100:00:171
1080999271,9cyclictest0-21swapper/122:16:541
1080999271,17cyclictest0-21swapper/122:36:551
1080999271,17cyclictest0-21swapper/122:10:351
1080999271,16cyclictest0-21swapper/122:20:061
1080999271,16cyclictest0-21swapper/121:32:291
1080399272,12cyclictest19310-21awk21:55:010
1082099266,17cyclictest14812-21head00:10:163
1082099263,4cyclictest28888-21latency_hist23:30:023
1082099263,21cyclictest39-21rcuc/319:37:583
1082099263,21cyclictest39-21rcuc/319:37:583
1082099262,21cyclictest5715-21/usr/sbin/munin20:10:153
1082099261,22cyclictest10313-21ps22:45:223
1082099261,18cyclictest0-21swapper/323:49:243
1080999269,9cyclictest0-21swapper/100:14:441
1080999266,10cyclictest25953-21fschecks_count20:55:161
1080999264,7cyclictest0-21swapper/123:08:511
1080999264,7cyclictest0-21swapper/123:08:511
1080999262,9cyclictest6337-21kworker/u8:222:40:501
1080999261,16cyclictest0-21swapper/121:27:531
1080999261,16cyclictest0-21swapper/121:18:151
1080999260,12cyclictest0-21swapper/119:45:381
1080399261,4cyclictest13548-21/usr/sbin/munin22:55:170
170102250,4sleep121-21rcuc/123:00:301
1082099251,9cyclictest22707-21needreboot22:00:183
1082099251,3cyclictest21989-21processes00:25:233
1082099251,21cyclictest32717-21/usr/sbin/munin22:25:163
1082099251,21cyclictest13568-21users00:05:273
1080999258,9cyclictest0-21swapper/121:20:271
10809992516,3cyclictest31775-21tail23:35:161
10809992513,7cyclictest0-21swapper/121:49:031
1080999251,19cyclictest10794-21cyclictest23:40:081
1080999251,16cyclictest0-21swapper/122:32:371
1080999251,12cyclictest0-21swapper/119:31:391
1080399257,15cyclictest1084-21perf20:00:010
1080399251,6cyclictest0-21swapper/023:00:150
1082099243,20cyclictest39-21rcuc/320:25:323
1082099241,4cyclictest32589-21latency_hist22:25:023
1082099241,3cyclictest0-21swapper/322:05:223
1082099241,2cyclictest20346-21kernelversion21:55:183
1082099241,2cyclictest20346-21kernelversion21:55:183
1082099241,20cyclictest20493-21/usr/sbin/munin19:30:193
1082099241,20cyclictest18682-21fschecks_count19:25:153
1082099241,20cyclictest0-21swapper/323:41:053
1082099241,11cyclictest5395-21smart_sda21:20:233
1082099241,10cyclictest0-21swapper/323:15:133
1082099240,2cyclictest0-21swapper/320:55:473
1080999243,15cyclictest26433-21kworker/u8:123:52:351
1080999243,15cyclictest18283-21kworker/u8:000:39:121
1080999242,7cyclictest10771-21kworker/u8:022:46:111
1080999241,20cyclictest178372sleep119:22:111
1080999241,20cyclictest0-21swapper/122:05:051
1080999241,16cyclictest0-21swapper/121:14:331
1080999241,15cyclictest790-21in:imjournal23:15:151
1080999241,15cyclictest0-21swapper/123:22:231
1080999241,15cyclictest0-21swapper/123:12:581
1080999241,11cyclictest0-21swapper/100:17:011
10809992410,7cyclictest0-21swapper/121:57:451
10809992410,7cyclictest0-21swapper/121:57:441
10809992410,6cyclictest0-21swapper/122:03:081
1080399243,2cyclictest87550irq/128-enp1s0-22:14:590
1080399241,10cyclictest28688-21head21:00:190
226892231,18sleep022842-21cpuspeed_turbos19:35:110
226892231,18sleep022842-21cpuspeed_turbos19:35:110
1082099231,9cyclictest0-21swapper/322:32:073
1082099231,8cyclictest909-21sssd_nss23:00:313
1082099231,3cyclictest0-21swapper/321:15:583
1082099231,14cyclictest0-21swapper/323:08:223
1082099231,14cyclictest0-21swapper/323:08:223
1082099231,11cyclictest1294-21/usr/sbin/munin20:00:113
1080999231,16cyclictest0-21swapper/120:00:341
1080999231,13cyclictest0-21swapper/120:11:471
1080999231,10cyclictest0-21swapper/121:40:351
1080999230,16cyclictest23621-21kworker/u8:122:26:581
1080999230,16cyclictest0-21swapper/121:02:371
1080399237,14cyclictest81ktimersoftd/022:00:010
1080399237,14cyclictest81ktimersoftd/022:00:010
1080399233,7cyclictest4385-40dnf22:33:240
1080399231,5cyclictest0-21swapper/023:55:490
1080399231,4cyclictest0-21swapper/021:20:140
43862220,5sleep21081599cyclictest23:45:222
1082099221,9cyclictest0-21swapper/300:35:183
1082099221,9cyclictest0-21swapper/300:16:353
1082099221,8cyclictest0-21swapper/300:24:003
1082099221,8cyclictest0-21swapper/300:24:003
1082099221,6cyclictest0-21swapper/322:15:313
1082099221,4cyclictest0-21swapper/322:51:333
1082099221,2cyclictest17710-21kworker/3:023:30:143
1082099221,2cyclictest0-21swapper/319:20:123
1082099221,14cyclictest0-21swapper/321:14:463
1082099221,10cyclictest0-21swapper/323:55:213
1081599224,5cyclictest25726-21processes23:20:232
1081599221,6cyclictest0-21swapper/223:30:122
1081599221,3cyclictest0-21swapper/220:32:242
10815992213,6cyclictest9946-21interrupts22:45:172
1080999223,2cyclictest16067-21kworker/u8:200:30:271
1080999221,16cyclictest0-21swapper/122:53:311
1080999221,16cyclictest0-21swapper/120:52:041
1080999221,15cyclictest0-21swapper/121:05:121
1080999221,15cyclictest0-21swapper/120:21:321
1080999221,15cyclictest0-21swapper/120:17:401
1080999221,13cyclictest0-21swapper/119:39:521
1080999221,13cyclictest0-21swapper/119:39:521
1080999221,11cyclictest579-21systemd-journal20:45:011
1080399222,11cyclictest24887-21/usr/sbin/munin19:40:170
1080399221,6cyclictest0-21swapper/023:09:500
1080399221,6cyclictest0-21swapper/023:09:500
1080399221,5cyclictest4158-21strings23:45:180
1080399221,4cyclictest0-21swapper/022:07:280
1080399221,12cyclictest23297-21/usr/sbin/munin20:50:230
1080399220,4cyclictest26433-21kworker/u8:123:24:140
1082099216,12cyclictest26224-21cpuspeed_turbos22:10:123
1082099211,6cyclictest0-21swapper/321:41:043
1082099211,4cyclictest0-21swapper/322:37:503
1082099211,2cyclictest0-21swapper/321:32:523
1082099211,2cyclictest0-21swapper/321:26:283
1082099211,11cyclictest0-21swapper/323:20:563
1082099211,11cyclictest0-21swapper/320:45:273
1082099211,11cyclictest0-21swapper/320:45:273
1082099211,10cyclictest18489-21NetworkManager20:37:423
1082099211,10cyclictest0-21swapper/321:35:233
1082099210,15cyclictest0-21swapper/300:00:193
1081599211,3cyclictest0-21swapper/220:27:362
1081599211,3cyclictest0-21swapper/220:02:002
1080999213,6cyclictest18296-21/usr/sbin/munin19:25:231
1080999211,18cyclictest0-21swapper/119:53:161
1080999211,15cyclictest0-21swapper/119:19:191
1080999211,14cyclictest0-21swapper/120:28:471
1080999211,14cyclictest0-21swapper/120:08:271
1080999211,12cyclictest0-21swapper/119:58:321
1080999211,12cyclictest0-21swapper/119:40:131
1080999211,10cyclictest11611-21gltestperf19:10:141
1080399212,11cyclictest15126-21/usr/sbin/munin21:45:220
1080399211,4cyclictest0-21swapper/000:36:100
10803992111,3cyclictest87550irq/128-enp1s0-22:20:020
1080399211,11cyclictest5366-21/usr/sbin/munin23:50:240
1080399211,11cyclictest30100-21kernelversion19:50:170
10803992110,3cyclictest12050irq/123-ahci[0023:20:010
1080399210,5cyclictest0-21swapper/000:30:040
147372200,4sleep121-21rcuc/120:30:131
1082099204,4cyclictest26115-21sendmail_mailqu19:40:233
1082099202,6cyclictest0-21swapper/322:40:033
10820992015,3cyclictest30760-21sed19:50:243
10820992014,3cyclictest32135-21gltestperf19:55:163
1082099201,10cyclictest0-21swapper/323:10:253
1082099201,10cyclictest0-21swapper/300:31:163
1081599202,6cyclictest0-21swapper/222:25:152
1081599201,7cyclictest0-21swapper/221:17:452
1081599201,7cyclictest0-21swapper/221:13:472
1081599201,6cyclictest18907-21/usr/sbin/munin20:40:132
1081599201,6cyclictest0-21swapper/221:45:012
1081599201,2cyclictest0-21swapper/222:21:282
1081599201,2cyclictest0-21swapper/219:34:402
1080999201,3cyclictest0-21swapper/120:35:221
1080999201,12cyclictest0-21swapper/120:48:081
1080999201,12cyclictest0-21swapper/120:48:081
1080399209,3cyclictest949-21/usr/sbin/munin23:40:200
1080399209,3cyclictest3521-21/usr/sbin/munin20:05:170
1080399209,3cyclictest21768-21strings00:25:200
1080399209,3cyclictest21141-21needreboot23:10:180
1080399208,2cyclictest0-21swapper/022:35:210
1080399205,6cyclictest755-21rngd23:35:250
1080399204,3cyclictest29064-21awk19:50:010
1080399202,10cyclictest12312-21/usr/sbin/munin20:25:120
1080399201,8cyclictest21482-21needreboot19:30:190
1080399201,11cyclictest11348-21/usr/sbin/munin22:50:160
1080399201,10cyclictest644-21cut21:10:190
1080399201,10cyclictest20095-21sed00:20:250
1080399201,10cyclictest20095-21sed00:20:250
10803992010,3cyclictest87550irq/128-enp1s0-21:35:000
1082099195,3cyclictest87650irq/129-enp1s0-20:05:183
1082099191,9cyclictest0-21swapper/323:50:263
1082099191,7cyclictest0-21swapper/320:51:523
1082099191,7cyclictest0-21swapper/319:46:533
1082099191,15cyclictest15787-21switchtime20:30:243
1082099191,15cyclictest0-21swapper/323:35:463
1081599194,4cyclictest87350irq/126-enp1s0-00:25:192
1081599193,3cyclictest87650irq/129-enp1s0-20:23:362
1081599192,4cyclictest18915-21users21:50:262
1081599191,9cyclictest0-21swapper/221:55:152
1081599191,9cyclictest0-21swapper/221:55:152
1081599191,7cyclictest3521-21/usr/sbin/munin20:05:152
1081599191,7cyclictest0-21swapper/221:08:512
1081599191,6cyclictest27698-21/usr/sbin/munin21:00:162
1081599191,6cyclictest0-21swapper/223:35:252
1081599191,6cyclictest0-21swapper/222:55:122
1081599191,6cyclictest0-21swapper/222:45:002
1081599191,6cyclictest0-21swapper/222:03:112
1081599191,6cyclictest0-21swapper/220:54:462
1081599191,5cyclictest0-21swapper/221:45:062
1081599191,5cyclictest0-21swapper/219:40:232
1081599191,5cyclictest0-21swapper/200:37:392
1081599191,4cyclictest22690-21/usr/sbin/munin19:35:232
1081599191,4cyclictest22690-21/usr/sbin/munin19:35:232
1081599191,3cyclictest0-21swapper/222:07:462
1080399198,3cyclictest808-21snmpd20:44:350
1080399198,3cyclictest21103-21/usr/sbin/munin20:45:160
1080399198,3cyclictest21103-21/usr/sbin/munin20:45:160
1080399196,4cyclictest12383-21sshd21:36:020
1080399191,8cyclictest13897-21/usr/sbin/munin19:15:100
1080399191,6cyclictest0-21swapper/022:41:050
1080399191,6cyclictest0-21swapper/021:15:220
1082099188,2cyclictest0-21swapper/319:15:113
1082099182,15cyclictest0-21swapper/320:41:163
1082099181,6cyclictest0-21swapper/320:23:523
1082099181,15cyclictest401ktimersoftd/320:15:303
1082099180,9cyclictest0-21swapper/319:10:223
1081599185,3cyclictest87350irq/126-enp1s0-19:25:262
1081599184,3cyclictest87350irq/126-enp1s0-23:50:212
1081599183,3cyclictest87550irq/128-enp1s0-20:10:372
1081599183,3cyclictest13774-21sshd00:07:472
1081599183,2cyclictest89050irq/133-enp2s0-23:00:162
1081599182,5cyclictest808-21snmpd19:59:262
1081599182,5cyclictest0-21swapper/222:10:082
1081599182,3cyclictest2902-21diskstats22:30:122
1081599181,9cyclictest0-21swapper/223:55:172
1081599181,6cyclictest766-21ntpd19:47:082
1081599181,6cyclictest0-21swapper/223:43:262
1081599181,6cyclictest0-21swapper/223:11:282
1081599181,6cyclictest0-21swapper/222:15:272
1081599181,6cyclictest0-21swapper/219:20:392
1081599181,5cyclictest5202-21ntp_states21:20:202
1081599181,5cyclictest25524-21cpu20:55:132
1081599181,5cyclictest18170-21expr23:05:122
1081599181,5cyclictest18170-21expr23:05:122
1081599181,5cyclictest0-21swapper/223:15:132
1081599181,5cyclictest0-21swapper/221:26:512
1081599181,5cyclictest0-21swapper/200:24:372
1081599181,5cyclictest0-21swapper/200:24:372
1081599181,5cyclictest0-21swapper/200:15:132
1081599181,5cyclictest0-21swapper/200:00:562
10815991813,3cyclictest26743-21acpi23:25:122
10815991812,3cyclictest14968-21ntp_states19:15:202
1080399189,2cyclictest0-21swapper/000:00:250
1080399187,3cyclictest808-21snmpd19:24:190
1080399186,4cyclictest808-21snmpd20:14:300
1080399185,5cyclictest30365-21latency_hist22:20:020
1080399183,3cyclictest87550irq/128-enp1s0-20:00:220
1080399181,8cyclictest808-21snmpd21:44:460
1080399181,6cyclictest0-21swapper/022:46:400
1080399181,6cyclictest0-21swapper/000:15:020
10803991813,3cyclictest14674-21tr20:30:130
1080399181,2cyclictest87350irq/126-enp1s0-23:26:510
1080399181,11cyclictest0-21swapper/000:10:270
1082099171,3cyclictest87650irq/129-enp1s0-21:00:163
1082099171,14cyclictest401ktimersoftd/321:05:133
1081599171,6cyclictest0-21swapper/222:37:522
1081599170,5cyclictest0-21swapper/221:35:132
1080399177,2cyclictest0-21swapper/020:55:230
1081599161,3cyclictest0-21swapper/220:45:182
1081599161,3cyclictest0-21swapper/220:45:182
1081599161,14cyclictest0-21swapper/220:36:202
1080399166,3cyclictest87550irq/128-enp1s0-20:35:230
1080399165,3cyclictest87550irq/128-enp1s0-23:30:150
1080399163,4cyclictest9084-21wc20:15:230
1080399161,7cyclictest0-21swapper/022:26:020
1080399161,5cyclictest0-21swapper/019:10:170
1080399161,4cyclictest0-21swapper/022:01:040
1080399161,12cyclictest29903-21/usr/sbin/munin21:05:210
10803991510,3cyclictest18591-21expr19:25:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional