You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-29 - 08:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Sun Mar 29, 2026 00:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
75972238163,25sleep30-21swapper/319:07:523
75972238163,25sleep30-21swapper/319:07:523
75692235163,59sleep00-21swapper/019:07:280
75692235163,59sleep00-21swapper/019:07:280
77272202165,25sleep20-21swapper/219:09:322
77272202165,25sleep20-21swapper/219:09:322
75112201163,25sleep10-21swapper/119:06:451
75112201163,25sleep10-21swapper/119:06:451
7866999793,2cyclictest28926-21diskmemload23:31:073
7866997874,3cyclictest0-21swapper/322:36:583
7866996359,3cyclictest0-21swapper/300:28:463
7866996259,2cyclictest0-21swapper/322:31:103
148682520,4sleep0784699cyclictest00:15:240
180942470,2sleep20-21swapper/223:10:172
7866994137,3cyclictest0-21swapper/321:45:493
786199371,14cyclictest20754-21idleruntime00:30:172
785499361,22cyclictest0-21swapper/123:00:151
784699369,13cyclictest4392-21munin-run23:55:010
7861993524,3cyclictest0-21swapper/220:45:252
786199351,26cyclictest0-21swapper/222:47:172
786199351,26cyclictest0-21swapper/222:47:172
786199351,20cyclictest0-21swapper/219:35:252
786199351,15cyclictest8475-21bash00:01:012
786199331,25cyclictest0-21swapper/222:25:572
786199331,19cyclictest0-21swapper/221:17:052
784699331,16cyclictest23901-21munin-run19:45:010
786199321,19cyclictest0-21swapper/222:59:412
786199321,15cyclictest0-21swapper/220:56:052
786199321,14cyclictest582-21ntp_states22:30:212
786199321,14cyclictest0-21swapper/223:04:052
786199321,13cyclictest0-21swapper/223:30:532
784699324,13cyclictest14154-21munin-run21:50:010
786199311,24cyclictest0-21swapper/200:24:092
786199311,23cyclictest0-21swapper/220:26:292
786199311,17cyclictest0-21swapper/223:27:332
786199311,17cyclictest0-21swapper/222:08:572
786199311,13cyclictest0-21swapper/223:21:132
786199311,13cyclictest0-21swapper/220:40:492
786199310,24cyclictest0-21swapper/200:05:072
785499311,24cyclictest0-21swapper/121:05:031
3232310,2sleep00-21swapper/021:15:260
94622300,4sleep339-21rcuc/322:50:203
7861993017,3cyclictest28926-21diskmemload22:51:132
7861993016,3cyclictest0-21swapper/222:19:252
786199301,14cyclictest0-21swapper/220:18:332
786199301,13cyclictest0-21swapper/223:42:292
786199301,13cyclictest0-21swapper/222:14:212
786199301,12cyclictest8847-21ntp_offset21:35:212
786199301,12cyclictest0-21swapper/221:01:322
785499300,18cyclictest0-21swapper/120:58:591
784699301,14cyclictest6606-21basename00:00:010
786199299,3cyclictest0-21swapper/223:15:202
786199295,10cyclictest5973-21fschecks_count21:30:152
786199295,10cyclictest5973-21fschecks_count21:30:152
7861992916,4cyclictest0-21swapper/221:48:532
7861992915,4cyclictest0-21swapper/219:32:492
786199291,23cyclictest0-21swapper/221:56:212
786199291,13cyclictest0-21swapper/221:50:492
786199291,11cyclictest0-21swapper/223:05:042
786199291,11cyclictest0-21swapper/221:10:492
785499291,19cyclictest24245-21ntp_states22:10:221
785499290,13cyclictest0-21swapper/121:35:191
786199281,14cyclictest0-21swapper/220:36:292
786199281,13cyclictest0-21swapper/219:16:242
786199281,13cyclictest0-21swapper/200:15:372
786199281,12cyclictest0-21swapper/222:02:132
786199281,12cyclictest0-21swapper/219:41:452
786199281,11cyclictest0-21swapper/200:14:052
785499281,9cyclictest0-21swapper/121:32:431
785499281,9cyclictest0-21swapper/121:32:431
785499281,8cyclictest0-21swapper/100:04:191
785499281,11cyclictest19740-21users20:45:271
785499281,11cyclictest0-21swapper/119:18:311
785499281,11cyclictest0-21swapper/100:18:351
784699281,13cyclictest10025-21munin-run19:15:010
7846992811,15cyclictest19825-21awk00:30:010
786199279,3cyclictest0-21swapper/220:30:212
786199271,15cyclictest0-21swapper/219:22:452
786199271,15cyclictest0-21swapper/219:11:572
786199271,13cyclictest0-21swapper/221:21:252
786199271,12cyclictest0-21swapper/220:11:412
786199271,12cyclictest0-21swapper/219:55:092
786199271,11cyclictest0-21swapper/223:54:372
786199271,11cyclictest0-21swapper/222:23:212
786199271,11cyclictest0-21swapper/200:35:292
785499271,9cyclictest0-21swapper/120:00:191
785499271,10cyclictest0-21swapper/123:25:031
785499270,12cyclictest0-21swapper/120:37:231
784699279,15cyclictest8812-21munin-run00:05:010
784699271,13cyclictest5360-21munin-run21:30:010
182032270,4sleep230-21rcuc/200:25:142
786199261,14cyclictest27240-21memory19:50:192
786199261,14cyclictest0-21swapper/222:39:212
786199261,13cyclictest0-21swapper/223:46:212
786199261,12cyclictest0-21swapper/223:36:492
785499261,10cyclictest0-21swapper/119:57:191
785499260,9cyclictest0-21swapper/120:20:071
785499260,19cyclictest0-21swapper/123:06:211
785499260,12cyclictest0-21swapper/122:52:021
785499260,10cyclictest0-21swapper/121:19:351
786699252,12cyclictest15115-21munin-run19:25:013
786199251,12cyclictest0-21swapper/222:42:172
786199251,11cyclictest3149-21kworker/u8:121:28:532
786199251,11cyclictest0-21swapper/221:41:372
786199251,10cyclictest0-21swapper/220:54:092
785499258,3cyclictest0-21swapper/119:11:111
785499257,3cyclictest0-21swapper/119:33:391
785499251,11cyclictest0-21swapper/119:28:551
785499250,9cyclictest0-21swapper/122:21:111
785499250,9cyclictest0-21swapper/122:03:231
785499250,9cyclictest0-21swapper/120:34:261
785499250,8cyclictest0-21swapper/123:54:471
785499250,12cyclictest0-21swapper/121:44:271
785499250,10cyclictest0-21swapper/123:43:271
786199248,3cyclictest0-21swapper/219:25:212
785499242,19cyclictest26670-21diskstats19:50:141
785499241,9cyclictest0-21swapper/120:51:191
785499241,9cyclictest0-21swapper/119:48:511
785499241,9cyclictest0-21swapper/119:44:511
785499241,10cyclictest0-21swapper/121:23:391
785499241,10cyclictest0-21swapper/120:29:471
7854992410,4cyclictest20077-21diskstats19:35:141
7854992410,2cyclictest0-21swapper/121:02:271
785499240,8cyclictest0-21swapper/122:40:541
785499240,10cyclictest0-21swapper/121:53:071
784699247,14cyclictest9-21rcu_preempt20:25:010
7846992411,7cyclictest4319-21ls22:40:130
786199239,3cyclictest0-21swapper/221:05:172
786199239,3cyclictest0-21swapper/220:24:532
786199237,10cyclictest0-21swapper/220:01:452
786199234,3cyclictest0-21swapper/220:05:172
786199231,11cyclictest0-21swapper/223:55:212
785499234,3cyclictest0-21swapper/123:57:551
785499233,3cyclictest0-21swapper/121:48:031
785499231,3cyclictest5307-21diskstats20:15:151
785499231,16cyclictest0-21swapper/121:57:251
785499231,12cyclictest6265-21cpuspeed_turbos22:45:111
785499231,12cyclictest6265-21cpuspeed_turbos22:45:111
785499231,12cyclictest30738-21users21:10:271
785499231,11cyclictest0-21swapper/100:29:111
784699231,11cyclictest15102-21idleruntime-cro19:25:010
786699222,11cyclictest32585-21/usr/sbin/munin23:45:133
7866992218,3cyclictest0-21swapper/320:10:463
786699221,18cyclictest22891-21switchtime23:20:263
786699221,11cyclictest809-21sshd22:45:163
786699221,11cyclictest809-21sshd22:45:163
786199221,2cyclictest0-21swapper/219:45:292
785499225,4cyclictest1045-21kernelversion23:45:191
7854992215,4cyclictest28298-21tr23:35:131
785499221,11cyclictest16902-21/usr/sbin/munin20:40:201
7854992210,3cyclictest26341-21diskstats23:30:141
785499220,8cyclictest0-21swapper/100:11:231
785499220,7cyclictest0-21swapper/100:20:591
785499220,11cyclictest0-21swapper/123:20:231
227292221,3sleep00-21swapper/022:07:240
786699211,7cyclictest0-21swapper/300:08:473
786699211,11cyclictest15956-21/usr/sbin/munin20:40:253
786699211,10cyclictest8236-21ntp_states20:20:213
785499217,3cyclictest0-21swapper/120:10:151
785499213,3cyclictest0-21swapper/121:26:221
785499211,9cyclictest0-21swapper/122:07:511
785499211,9cyclictest0-21swapper/100:07:151
7854992115,4cyclictest12611-21cat23:00:011
785499210,10cyclictest0-21swapper/122:36:421
784699217,6cyclictest833-21runrttasks21:36:110
784699215,4cyclictest11849-21grep00:10:150
784699211,6cyclictest0-21swapper/023:02:280
784699211,11cyclictest27315-21strings19:50:190
784699211,11cyclictest24742-21grep19:45:150
784699211,11cyclictest13756-21/usr/sbin/munin20:35:120
786699208,3cyclictest5205-21sshd21:28:153
786699201,11cyclictest10601-21/usr/sbin/munin22:55:143
785499209,3cyclictest19744-21diskstats23:15:141
785499204,2cyclictest0-21swapper/100:35:111
785499201,14cyclictest0-21swapper/122:15:341
785499201,10cyclictest19167-21latency_hist23:15:021
785499201,10cyclictest0-21swapper/122:33:471
784699208,3cyclictest24739-21/usr/sbin/munin21:00:190
784699206,5cyclictest0-21swapper/000:35:470
784699206,2cyclictest86550irq/126-enp1s0-00:23:390
784699204,5cyclictest0-21swapper/023:12:420
7846992015,3cyclictest24015-21grep20:55:260
7846992014,3cyclictest21362-21crond23:20:010
784699201,11cyclictest4960-21/usr/sbin/munin20:15:240
784699201,11cyclictest25993-21/usr/sbin/munin23:30:200
784699201,11cyclictest25369-21/usr/sbin/munin22:15:170
784699201,11cyclictest0-21swapper/000:05:460
784699201,10cyclictest0-21swapper/022:34:560
784699200,4cyclictest29558-21basename22:25:010
786699198,3cyclictest26323-21/usr/sbin/munin19:50:243
786699198,3cyclictest20038-21/usr/sbin/munin00:30:163
786699194,12cyclictest41-21ksoftirqd/321:40:113
786699193,7cyclictest39-21rcuc/321:10:173
786699192,3cyclictest0-21swapper/323:35:153
786699192,14cyclictest30423-21sed22:25:163
786699191,9cyclictest0-21swapper/320:16:153
786699191,8cyclictest461-21munin-node20:05:123
786699191,6cyclictest0-21swapper/300:17:323
7866991914,4cyclictest0-21swapper/320:27:163
7866991914,3cyclictest13035-21grep20:30:253
786699191,15cyclictest25271-21grep23:25:253
786699191,15cyclictest22274-21diskstats19:40:143
786699191,12cyclictest0-21swapper/321:05:113
786699191,11cyclictest0-21swapper/319:10:263
786699191,10cyclictest19782-21fschecks_count23:15:153
786699191,10cyclictest0-21swapper/321:15:193
786699191,10cyclictest0-21swapper/319:49:163
785499195,3cyclictest0-21swapper/119:20:031
7854991915,3cyclictest0-21swapper/122:29:521
785499191,10cyclictest20053-21acpi00:30:121
785499190,17cyclictest0-21swapper/120:05:241
784699199,3cyclictest833-21runrttasks21:59:250
784699197,4cyclictest0-21swapper/022:47:300
784699197,4cyclictest0-21swapper/022:47:300
784699197,4cyclictest0-21swapper/022:02:300
784699196,3cyclictest0-21swapper/022:58:370
784699195,2cyclictest11196-21processes21:40:220
784699195,13cyclictest0-21swapper/020:05:010
784699192,3cyclictest15048-21hddtemp_smartct21:50:160
784699191,9cyclictest16327-21strings19:25:200
784699191,5cyclictest0-21swapper/000:34:120
784699191,10cyclictest18152-21/usr/sbin/munin20:45:250
784699190,8cyclictest0-21swapper/023:28:200
250912191,2sleep00-21swapper/022:14:300
786699189,7cyclictest86550irq/126-enp1s0-23:50:103
786699187,3cyclictest18825-21cat19:30:223
786699181,7cyclictest6597-21perf00:00:013
786699181,6cyclictest4473-21fschecks_time22:40:153
7866991813,3cyclictest8599-21irqstats21:35:183
7866991813,3cyclictest15045-21tail21:50:153
7866991812,4cyclictest20344-21grep19:35:153
786699181,15cyclictest0-21swapper/300:11:013
786699181,10cyclictest0-21swapper/322:15:273
784699187,3cyclictest0-21swapper/021:30:090
784699187,3cyclictest0-21swapper/021:30:090
784699184,3cyclictest86750irq/128-enp1s0-19:15:170
784699183,4cyclictest9914-21grep22:50:250
784699182,7cyclictest17534-21/usr/sbin/munin19:30:130
784699181,3cyclictest10355-21head20:25:190
784699181,15cyclictest0-21swapper/020:41:340
784699181,10cyclictest0-21swapper/021:20:060
784699180,8cyclictest0-21swapper/023:36:250
784699180,8cyclictest0-21swapper/022:38:090
232392181,2sleep00-21swapper/023:21:000
215652181,3sleep00-21swapper/019:38:270
786699172,12cyclictest10943-21/usr/sbin/munin19:15:273
786699171,9cyclictest0-21swapper/300:35:253
786699171,7cyclictest17201-21/usr/sbin/munin23:10:173
786699171,7cyclictest16876-21sendmail_mailqu00:20:243
786699171,7cyclictest0-21swapper/323:05:193
786699171,6cyclictest0-21swapper/322:24:523
786699171,6cyclictest0-21swapper/322:10:153
786699171,6cyclictest0-21swapper/321:55:513
786699171,5cyclictest0-21swapper/322:05:433
7866991712,3cyclictest8323-21unixbench_multi00:00:263
786699171,13cyclictest401ktimersoftd/323:40:173
7866991711,3cyclictest19226-21cut22:00:153
786699170,15cyclictest0-21swapper/321:21:413
784699178,2cyclictest0-21swapper/022:25:120
784699176,9cyclictest86550irq/126-enp1s0-20:30:210
784699176,3cyclictest1013-21irqstats23:45:180
784699171,3cyclictest29757-21head21:10:160
784699171,14cyclictest0-21swapper/021:05:490
7846991711,3cyclictest31692-21switchtime23:40:230
324152171,3sleep30-21swapper/320:01:343
155052171,2sleep30-21swapper/320:37:223
786699162,4cyclictest6847-21smart_sda21:30:233
786699162,4cyclictest6847-21smart_sda21:30:233
786699161,6cyclictest0-21swapper/323:00:073
786699161,6cyclictest0-21swapper/319:26:113
786699161,3cyclictest0-21swapper/319:56:303
7866991612,2cyclictest25415-21hddtemp_smartct21:00:163
786699161,13cyclictest0-21swapper/320:55:133
7866991610,4cyclictest0-21swapper/320:50:143
784699169,4cyclictest0-21swapper/023:05:230
784699164,5cyclictest0-21swapper/020:50:170
784699161,14cyclictest0-21swapper/019:56:450
7846991611,3cyclictest2294-21seq20:07:070
786699155,2cyclictest86550irq/126-enp1s0-20:45:233
7846991510,3cyclictest4665-21seq20:14:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional