You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-07 - 08:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Tue Jul 07, 2026 00:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
41192241164,24sleep20-21swapper/219:09:342
38932238165,25sleep00-21swapper/019:06:380
38842236163,24sleep10-21swapper/119:06:301
38912203166,24sleep30-21swapper/319:06:373
2027821650,4sleep3426299cyclictest23:26:353
36482750,4sleep1425299cyclictest21:35:231
4257995956,2cyclictest0-21swapper/222:10:432
193872560,4sleep2425799cyclictest21:00:012
149492530,1sleep121-21rcuc/100:30:141
297952520,4sleep339-21rcuc/322:35:183
246392430,1sleep30-21swapper/321:10:183
40082420,3sleep24011-21latency_hist20:25:012
424799379,13cyclictest23722-21diskmemload23:25:000
43382330,18sleep20-21swapper/222:50:212
4252992912,15cyclictest23150-21kworker/u8:219:43:131
4252992912,15cyclictest23150-21kworker/u8:219:43:131
4252992910,6cyclictest14223-40dnf19:34:051
425299285,21cyclictest23-21ksoftirqd/100:00:071
426299272,16cyclictest23726-21munin-run21:10:003
425299271,20cyclictest0-21swapper/120:17:101
424799272,12cyclictest23103-21munin-run19:55:010
426299260,15cyclictest1682-21switchtime21:30:273
425299261,21cyclictest25316-21cpuspeed_turbos23:40:121
425299260,15cyclictest0-21swapper/121:15:291
189582260,4sleep339-21rcuc/319:45:103
426299257,3cyclictest17430-21awk00:40:013
426299251,16cyclictest41-21ksoftirqd/300:23:183
426299251,16cyclictest41-21ksoftirqd/300:23:183
425299251,6cyclictest0-21swapper/121:47:411
4252992512,11cyclictest16190-21kworker/u8:122:15:291
425299251,16cyclictest10116-21sed20:35:261
4252992510,13cyclictest27118-21kworker/u8:123:56:471
425299250,10cyclictest23722-21diskmemload22:10:271
4247992514,3cyclictest0-21swapper/023:25:270
424799251,12cyclictest17758-21awk22:10:010
426299248,10cyclictest87350irq/126-enp1s0-22:25:273
426299246,3cyclictest41-21ksoftirqd/323:45:023
426299241,15cyclictest0-21swapper/321:50:163
426299241,12cyclictest0-21swapper/323:09:113
425299249,4cyclictest0-21swapper/122:45:071
425299247,4cyclictest0-21swapper/123:27:411
425299242,3cyclictest18941-21users20:55:281
425299241,16cyclictest0-21swapper/122:41:141
424799240,11cyclictest10208-21idleruntime-cro00:20:010
4262992310,5cyclictest401ktimersoftd/322:40:133
425799231,5cyclictest0-21swapper/223:25:282
425299238,9cyclictest0-21swapper/100:28:431
425299231,5cyclictest0-21swapper/123:00:441
425299231,2cyclictest17236-21sshd00:37:541
425299231,16cyclictest0-21swapper/122:31:551
425299231,16cyclictest0-21swapper/122:27:341
425299231,12cyclictest0-21swapper/122:20:211
425299231,11cyclictest0-21swapper/122:07:581
4252992310,11cyclictest8679-21kworker/u8:022:37:581
4247992311,3cyclictest15322-21kworker/0:022:26:310
51162221,3sleep15127-21diskstats21:40:141
426299228,3cyclictest6364-21/usr/sbin/munin20:30:213
426299221,6cyclictest0-21swapper/323:50:123
426299221,6cyclictest0-21swapper/322:05:223
426299221,5cyclictest0-21swapper/322:21:323
4262992211,3cyclictest1445-21fschecks_time22:45:143
4262992211,3cyclictest12050irq/123-ahci[0021:25:023
426299221,11cyclictest0-21swapper/323:01:023
426299221,10cyclictest0-21swapper/300:10:383
4262992210,3cyclictest87350irq/126-enp1s0-22:20:023
425799221,5cyclictest0-21swapper/221:16:262
425299227,10cyclictest0-21swapper/123:09:201
4252992215,2cyclictest0-21swapper/120:04:091
425299221,18cyclictest28991-21/usr/sbin/munin20:05:241
425299221,15cyclictest0-21swapper/121:50:181
425299221,15cyclictest0-21swapper/100:21:101
425299221,15cyclictest0-21swapper/100:21:101
425299221,14cyclictest0-21swapper/100:07:281
425299221,11cyclictest31760-21gunzip21:25:221
426299218,4cyclictest0-21swapper/322:52:133
426299214,7cyclictest87350irq/126-enp1s0-19:55:203
426299213,7cyclictest401ktimersoftd/320:05:013
426299212,8cyclictest87350irq/126-enp1s0-21:30:023
425799211,5cyclictest0-21swapper/222:38:092
425799211,17cyclictest22763-21perf23:35:022
4252992113,3cyclictest0-21swapper/121:12:191
4252992113,3cyclictest0-21swapper/120:21:281
4252992113,3cyclictest0-21swapper/120:11:571
4252992113,3cyclictest0-21swapper/119:24:281
425299211,18cyclictest0-21swapper/122:00:561
425299211,18cyclictest0-21swapper/122:00:561
425299211,17cyclictest13306-21/usr/sbin/munin20:45:161
4252992111,5cyclictest0-21swapper/123:16:111
425299211,15cyclictest0-21swapper/123:13:251
425299211,15cyclictest0-21swapper/123:13:251
425299211,15cyclictest0-21swapper/120:52:511
425299211,15cyclictest0-21swapper/120:52:511
425299211,15cyclictest0-21swapper/120:40:581
425299211,15cyclictest0-21swapper/119:36:041
425299211,15cyclictest0-21swapper/119:13:331
425299211,14cyclictest87550irq/128-enp1s0-23:50:201
425299211,14cyclictest0-21swapper/119:26:001
425299211,11cyclictest6991-21users22:55:311
425299210,15cyclictest0-21swapper/120:30:271
424799212,7cyclictest10240-21ntp_states21:50:210
424799211,17cyclictest24342-21munin-run22:25:020
424799211,11cyclictest4842-21cpuspeed_turbos21:40:120
424799210,9cyclictest27890-21sshd21:17:150
426299208,4cyclictest839-21lldpd20:05:133
426299206,3cyclictest87650irq/129-enp1s0-00:05:153
426299204,9cyclictest87350irq/126-enp1s0-21:35:203
426299202,11cyclictest0-21swapper/323:55:183
426299201,8cyclictest41-21ksoftirqd/323:35:163
426299201,11cyclictest14874-21df19:35:133
426299200,9cyclictest0-21swapper/321:42:293
425799209,3cyclictest13354-21awk22:00:012
425799207,3cyclictest87350irq/126-enp1s0-19:20:212
425799201,7cyclictest0-21swapper/223:57:202
425799201,6cyclictest0-21swapper/223:24:412
425799201,6cyclictest0-21swapper/200:07:352
425799201,11cyclictest0-21swapper/223:40:262
425799201,11cyclictest0-21swapper/221:30:482
425299202,15cyclictest21532-21munin-run21:05:011
425299202,10cyclictest909-21sssd_nss21:05:221
4252992015,3cyclictest19698-21if_err_enp1s019:45:141
4252992013,2cyclictest0-21swapper/119:19:571
425299201,17cyclictest0-21swapper/123:33:031
425299201,14cyclictest0-21swapper/123:22:441
425299201,14cyclictest0-21swapper/122:51:041
425299201,14cyclictest0-21swapper/121:21:101
425299201,14cyclictest0-21swapper/120:29:111
425299201,14cyclictest0-21swapper/119:56:521
425299201,10cyclictest0-21swapper/121:30:411
425299200,2cyclictest0-21swapper/123:39:281
425299200,14cyclictest0-21swapper/100:19:001
425299200,11cyclictest0-21swapper/100:10:191
424799208,11cyclictest0-21swapper/020:55:010
424799208,11cyclictest0-21swapper/020:55:010
424799202,11cyclictest28956-21/usr/sbin/munin22:35:130
424799201,5cyclictest18245-1kworker/u9:219:31:480
424799201,10cyclictest9104-21unixbench_multi23:00:260
424799201,10cyclictest11785-21awk23:10:010
426299198,2cyclictest12129-21awk19:30:013
426299195,3cyclictest0-21swapper/323:46:013
426299193,3cyclictest12358-21needreboot21:55:213
426299192,9cyclictest41-21ksoftirqd/322:01:523
426299192,9cyclictest41-21ksoftirqd/322:01:523
426299192,6cyclictest0-21swapper/300:16:113
426299192,2cyclictest41-21ksoftirqd/321:15:173
426299192,11cyclictest0-21swapper/300:27:503
426299192,11cyclictest0-21swapper/300:02:443
4262991913,4cyclictest21384-21grep23:30:163
426299191,13cyclictest0-21swapper/322:30:273
426299191,10cyclictest7891-21memory21:45:193
426299191,10cyclictest10759-21/usr/sbin/munin20:40:253
426299190,9cyclictest0-21swapper/323:20:323
426299190,9cyclictest0-21swapper/323:19:413
425799192,7cyclictest24602-40dnf23:35:382
425799192,14cyclictest3220-21cut21:35:192
425799191,6cyclictest0-21swapper/222:48:132
425799191,6cyclictest0-21swapper/220:29:312
425799191,4cyclictest8566-21users21:45:272
425299194,9cyclictest0-21swapper/123:45:431
4252991912,2cyclictest0-21swapper/121:57:321
4252991911,2cyclictest0-21swapper/119:50:401
424799191,9cyclictest0-21swapper/000:24:000
424799191,9cyclictest0-21swapper/000:24:000
424799191,7cyclictest0-21swapper/023:17:030
424799191,6cyclictest0-21swapper/000:06:560
424799191,10cyclictest6045-21/usr/sbin/munin00:10:160
424799191,10cyclictest25183-21/usr/sbin/munin23:40:250
424799191,10cyclictest23953-21/usr/sbin/munin21:10:220
426299187,5cyclictest87650irq/129-enp1s0-20:15:213
426299187,4cyclictest16812-21awk00:35:013
426299183,7cyclictest12050irq/123-ahci[0020:40:013
426299183,6cyclictest87350irq/126-enp1s0-20:55:443
426299182,3cyclictest0-21swapper/321:04:173
426299181,6cyclictest0-21swapper/322:14:513
4262991814,3cyclictest0-21swapper/320:21:243
426299181,2cyclictest808-21snmpd20:27:453
4262991812,3cyclictest12105-21cpuspeed_turbos23:10:113
4262991812,3cyclictest12105-21cpuspeed_turbos23:10:113
426299181,11cyclictest0-21swapper/322:55:293
425799182,7cyclictest17068-21sed22:05:232
425799181,8cyclictest9561-21fschecks_count21:50:152
425799181,7cyclictest6162-21cpuspeed_turbos00:10:132
425799181,6cyclictest9584-21perf23:05:012
425799181,5cyclictest0-21swapper/221:03:352
425799181,2cyclictest0-21swapper/222:57:322
4257991812,3cyclictest31316-21cpuspeed_turbos22:40:122
425799181,15cyclictest29419-21ntp_states21:20:242
425799181,15cyclictest0-21swapper/200:23:022
425799181,15cyclictest0-21swapper/200:23:022
4257991811,4cyclictest87350irq/126-enp1s0-19:15:022
425799181,11cyclictest0-21swapper/221:11:312
425799181,10cyclictest0-21swapper/222:24:572
425799181,10cyclictest0-21swapper/221:42:482
424799186,3cyclictest21777-21needreboot23:30:190
424799186,3cyclictest20897-21idleruntime-cro19:50:010
424799184,8cyclictest0-21swapper/000:28:180
424799183,8cyclictest87550irq/128-enp1s0-23:55:140
424799181,9cyclictest808-21snmpd23:37:410
424799181,7cyclictest28724-21grep20:05:200
424799181,7cyclictest0-21swapper/022:16:350
4247991814,2cyclictest7353-21strings20:30:190
424799181,10cyclictest0-21swapper/021:20:030
4247991810,2cyclictest87550irq/128-enp1s0-22:32:110
424799180,11cyclictest0-21swapper/023:10:140
424799180,11cyclictest0-21swapper/023:10:140
34112180,2sleep00-21swapper/020:20:270
243432181,2sleep20-21swapper/222:25:012
96922170,3sleep30-21swapper/319:22:153
426299178,2cyclictest12050irq/123-ahci[0019:35:013
426299176,5cyclictest87350irq/126-enp1s0-20:45:133
426299172,3cyclictest18701-21sh19:45:013
426299172,3cyclictest18701-21sh19:45:013
426299171,2cyclictest30921-21strings20:10:193
426299171,13cyclictest22125-21cut19:50:193
4262991710,4cyclictest909-21sssd_nss20:50:153
4262991710,4cyclictest909-21sssd_nss20:50:153
425799175,3cyclictest32-21ksoftirqd/221:25:112
425799175,3cyclictest12050irq/123-ahci[0019:55:022
425799171,9cyclictest0-21swapper/220:45:142
425799171,7cyclictest1100-21cat20:15:232
425799171,5cyclictest27516-21awk20:05:002
425799171,5cyclictest0-21swapper/223:14:302
425799171,5cyclictest0-21swapper/223:14:302
425799171,3cyclictest0-21swapper/223:15:552
4257991712,3cyclictest28055-21hddtemp_smartct23:45:162
425799171,14cyclictest0-21swapper/200:39:092
425799171,14cyclictest0-21swapper/200:25:062
4257991711,3cyclictest7432-21ntp_states20:30:212
4257991711,3cyclictest20528-21entropy22:15:152
425799171,10cyclictest0-21swapper/220:44:462
425799171,10cyclictest0-21swapper/219:12:112
425799170,15cyclictest0-21swapper/220:10:172
425799170,15cyclictest0-21swapper/220:06:352
424799176,3cyclictest4345-21grep22:50:220
424799175,3cyclictest87550irq/128-enp1s0-21:45:150
424799173,8cyclictest0-21swapper/021:33:170
424799172,6cyclictest0-21swapper/022:55:240
424799172,5cyclictest1731-21sh20:20:020
424799171,9cyclictest808-21snmpd21:27:260
424799171,9cyclictest0-21swapper/000:02:120
424799171,8cyclictest12163-21if_err_enp2s021:55:190
424799171,6cyclictest0-21swapper/022:41:000
424799171,6cyclictest0-21swapper/022:14:470
424799171,5cyclictest0-21swapper/022:00:250
424799171,5cyclictest0-21swapper/022:00:250
424799171,3cyclictest31509-21sshd23:54:370
424799171,2cyclictest87650irq/129-enp1s0-19:25:190
4247991712,3cyclictest8976-21netstat19:20:220
4247991711,3cyclictest4978-21munin-node19:15:030
424799171,13cyclictest30923-21head20:10:190
424799171,10cyclictest0-21swapper/022:45:150
4247991710,3cyclictest4510-21df_inode20:25:150
424799170,5cyclictest0-21swapper/019:35:220
293012170,2sleep00-21swapper/023:49:310
271272170,2sleep00-21swapper/020:00:290
45352160,2sleep00-21swapper/019:11:250
4262991613,2cyclictest41-21ksoftirqd/319:15:023
426299161,12cyclictest5749-21/usr/sbin/munin19:15:203
425799167,6cyclictest16842-21cpuspeed_turbos19:40:132
425799167,6cyclictest16842-21cpuspeed_turbos19:40:132
425799165,3cyclictest87650irq/129-enp1s0-20:35:172
425799165,2cyclictest24092-21if_enp2s019:55:182
425799162,5cyclictest2692-21ntp_states00:00:202
425799161,9cyclictest0-21swapper/200:15:432
425799161,5cyclictest0-21swapper/222:30:142
425799161,4cyclictest909-21sssd_nss23:05:182
4257991612,3cyclictest0-21swapper/223:50:212
4257991612,3cyclictest0-21swapper/221:07:052
4257991612,3cyclictest0-21swapper/220:52:092
4257991612,3cyclictest0-21swapper/220:52:092
4257991611,3cyclictest15871-21ntp_kernel_pll_00:30:212
4257991611,3cyclictest12676-21df19:30:132
4257991610,3cyclictest19570-21gltestperf19:45:142
4257991610,3cyclictest14713-21seq19:35:132
4257991610,3cyclictest13789-21cat22:00:132
4257991610,3cyclictest13789-21cat22:00:132
424799166,3cyclictest87350irq/126-enp1s0-21:05:150
424799164,11cyclictest87350irq/126-enp1s0-20:55:180
424799162,4cyclictest13946-21grep20:45:210
424799161,5cyclictest0-21swapper/021:35:130
424799161,3cyclictest8566-21/usr/sbin/munin20:35:130
4247991612,3cyclictest0-21swapper/000:35:040
424799161,14cyclictest0-21swapper/020:43:190
424799161,14cyclictest0-21swapper/000:30:550
4247991611,3cyclictest20539-21strings21:00:200
4247991610,3cyclictest827-21runrttasks19:56:290
425799159,3cyclictest10661-21cut19:25:162
424799152,10cyclictest17333-21grep19:40:160
424799152,10cyclictest17333-21grep19:40:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional