You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-22 - 09:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Mon Jun 22, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
28590210086,9sleep00-21swapper/019:08:550
2851029279,9sleep20-21swapper/219:07:532
2844228665,6sleep10-21swapper/119:06:591
2707528369,9sleep30-21swapper/319:05:173
42222510,0sleep30-21swapper/323:14:443
28761994443,1cyclictest311rcuc/221:41:102
28761994321,21cyclictest311rcuc/222:37:132
28761994221,21cyclictest311rcuc/223:17:122
28761994221,21cyclictest311rcuc/223:11:072
28761994221,21cyclictest311rcuc/200:27:012
28761994221,20cyclictest311rcuc/223:58:142
28761994220,21cyclictest311rcuc/223:53:512
28761994220,21cyclictest311rcuc/223:40:352
28761994220,21cyclictest311rcuc/223:38:442
28761994220,21cyclictest311rcuc/223:05:162
28761994220,21cyclictest311rcuc/222:10:372
28761994220,21cyclictest311rcuc/200:04:272
28761994121,20cyclictest311rcuc/200:07:402
28761994120,21cyclictest311rcuc/222:54:372
28761994120,21cyclictest311rcuc/222:48:002
28761994120,21cyclictest311rcuc/222:31:472
28761994120,21cyclictest311rcuc/221:57:212
28761994120,21cyclictest311rcuc/221:34:592
28761994120,21cyclictest311rcuc/221:23:092
28761994120,21cyclictest311rcuc/200:36:432
28761994120,21cyclictest311rcuc/200:31:312
28761994120,21cyclictest311rcuc/200:17:542
28761994120,21cyclictest311rcuc/200:10:462
28758994120,20cyclictest38150irq/131-snd_hda23:22:031
28761994020,20cyclictest311rcuc/221:26:072
28761994019,21cyclictest311rcuc/223:45:372
28761994019,21cyclictest311rcuc/223:25:212
28761994019,21cyclictest311rcuc/222:15:372
28761994019,21cyclictest311rcuc/221:53:342
28761994019,21cyclictest311rcuc/221:35:392
28761994019,21cyclictest311rcuc/221:19:042
28761994019,21cyclictest311rcuc/221:11:182
28761994019,20cyclictest311rcuc/222:05:122
28761994018,21cyclictest311rcuc/223:33:202
28761993919,20cyclictest311rcuc/223:03:232
28761993919,20cyclictest311rcuc/222:41:472
28758993918,20cyclictest38150irq/131-snd_hda00:20:351
28764993816,21cyclictest0-21swapper/322:10:083
28758993719,17cyclictest38150irq/131-snd_hda21:38:151
28758993715,21cyclictest19201-21kworker/1:223:44:011
28758993616,20cyclictest38150irq/131-snd_hda23:25:411
28758993615,21cyclictest38150irq/131-snd_hda00:09:061
28758993615,20cyclictest55850irq/129-enp2s0-22:50:311
28764993514,20cyclictest55650irq/127-enp2s0-00:10:313
28761993514,21cyclictest311rcuc/222:29:532
28758993513,21cyclictest38150irq/131-snd_hda23:57:561
28758993513,21cyclictest15326-21kworker/1:122:24:261
28764993413,20cyclictest55550irq/126-enp2s0-23:02:163
28764993412,21cyclictest55550irq/126-enp2s0-22:15:353
28761993414,20cyclictest55850irq/129-enp2s0-22:59:522
28758993413,21cyclictest38150irq/131-snd_hda22:06:531
28758993412,21cyclictest38150irq/131-snd_hda22:01:421
2875499340,21cyclictest0-21swapper/000:23:530
28761993312,21cyclictest311rcuc/221:49:342
28758993312,20cyclictest241rcuc/121:46:551
28758993311,21cyclictest38150irq/131-snd_hda00:26:371
28764993211,20cyclictest55850irq/129-enp2s0-21:20:033
28761993220,11cyclictest311rcuc/223:20:212
28758993212,20cyclictest38150irq/131-snd_hda21:44:271
28758993211,21cyclictest38150irq/131-snd_hda23:38:541
28758993210,21cyclictest38150irq/131-snd_hda23:14:121
28758993210,21cyclictest38150irq/131-snd_hda22:30:131
2876499310,20cyclictest0-21swapper/322:04:073
2875899319,21cyclictest38150irq/131-snd_hda00:01:441
28758993110,21cyclictest38150irq/131-snd_hda23:32:331
2875899309,21cyclictest38150irq/131-snd_hda00:34:461
28758993013,16cyclictest38150irq/131-snd_hda22:49:131
28758993010,20cyclictest38150irq/131-snd_hda23:02:021
28758993010,20cyclictest38150irq/131-snd_hda00:11:501
2876499297,21cyclictest0-21swapper/300:31:453
2876199298,21cyclictest311rcuc/200:21:172
2875899298,21cyclictest38150irq/131-snd_hda22:27:321
2875499292,21cyclictest95632sleep000:34:590
2875499291,22cyclictest6155-21sed23:15:170
2875499291,22cyclictest11901-21fw_packets22:40:170
2875499291,21cyclictest537-21snmpd21:31:080
2875499291,21cyclictest455-21dbus-daemon00:10:040
2875499291,21cyclictest3100-21forks22:35:160
2875499291,21cyclictest25785-21apt-get00:25:150
2875499291,21cyclictest131602sleep022:02:210
2875499290,23cyclictest16778-21taskset21:25:290
2875499290,22cyclictest21984-21grep23:25:120
2875499290,21cyclictest141rcu_preempt23:04:080
2876499280,27cyclictest21866-21ssh21:10:123
2875899286,21cyclictest38150irq/131-snd_hda21:18:361
2875499283,21cyclictest13-21ksoftirqd/000:17:440
28754992814,13cyclictest0-21swapper/022:55:500
2875499281,22cyclictest537-21snmpd00:06:360
2875499281,22cyclictest28432-21sed22:50:150
2875499281,22cyclictest16965-21kernelversion00:00:220
2875499281,22cyclictest13540-21apache_processe23:20:130
2875499281,22cyclictest10586-21hddtemp_smartct22:20:170
2875499281,21cyclictest89972sleep021:21:010
2875499281,21cyclictest31381-21apache_processe23:50:110
2875499281,21cyclictest30335-21grep23:30:140
2875499281,21cyclictest14693-21apt-get23:40:140
2875499281,21cyclictest0-21swapper/023:10:490
2875499281,21cyclictest0-21swapper/022:45:220
2875499281,21cyclictest0-21swapper/022:12:540
2875499280,24cyclictest225-21systemd-journal22:08:230
2875499280,22cyclictest21854-21apt-get21:10:130
2875499280,22cyclictest10109-21cut00:35:130
2875499280,21cyclictest26812sleep021:55:440
2875499280,21cyclictest26660-21grep22:30:150
2875499280,21cyclictest0-21swapper/023:35:430
2876499274,22cyclictest32049-21diskstats21:35:163
2876499274,22cyclictest121002sleep323:38:253
2875499271,22cyclictest3378-21users22:15:270
2875499271,21cyclictest7366-21apt-get21:40:140
2875499271,21cyclictest30284-21apache_volume21:15:130
2875499271,21cyclictest21465-21grep23:05:170
2875499271,21cyclictest0-21swapper/021:45:350
2875499270,23cyclictest24749-21cstates21:50:160
2875499270,21cyclictest0-21swapper/023:56:400
2875499270,21cyclictest0-21swapper/023:48:300
2875499270,21cyclictest0-21swapper/022:29:170
2876499264,21cyclictest276132chrt22:50:043
2876499260,22cyclictest225-21systemd-journal21:51:123
2875899269,17cyclictest56550irq/124-enp0s3123:17:281
28758992620,3cyclictest38150irq/131-snd_hda22:42:001
2875899261,21cyclictest141rcu_preempt21:52:151
2875899260,22cyclictest49-21kcompactd021:22:011
2875899260,1cyclictest0-21swapper/121:56:201
2875899260,0cyclictest0-21swapper/121:30:261
2875499260,21cyclictest0-21swapper/021:36:210
2876499252,22cyclictest537-21snmpd23:27:073
2876499252,22cyclictest39-21ksoftirqd/323:22:173
2876499252,22cyclictest31169-21smartctl23:30:183
2876499252,22cyclictest26447-21/usr/sbin/munin22:30:153
2876499252,22cyclictest18061-21/usr/sbin/munin22:25:263
2876499252,22cyclictest11012-21apache200:07:003
2876499251,23cyclictest537-21snmpd23:15:073
2876499251,21cyclictest141rcu_preempt22:07:273
2876499251,20cyclictest141rcu_preempt21:56:003
2876499250,24cyclictest17307-21ntp_states21:45:203
2876499250,24cyclictest10047-21apache_accesses00:35:103
2876499250,23cyclictest10368-21forks22:20:153
2876199252,22cyclictest9332-21dnf22:00:162
2875899254,21cyclictest38150irq/131-snd_hda23:05:421
28758992523,1cyclictest25-21ksoftirqd/122:58:301
2875899251,22cyclictest14657-21apache_accesses21:25:111
2875899251,21cyclictest141rcu_preempt21:13:241
2875899251,21cyclictest141rcu_preempt00:15:191
2875899250,24cyclictest10052-21apache_accesses00:35:111
2875899250,24cyclictest0-21swapper/123:54:341
2876499241,22cyclictest8024-21fschecks_count23:55:163
2876499241,22cyclictest3762-21ntp_states22:35:213
2876499241,22cyclictest19445-21apache_accesses22:45:113
2876499241,22cyclictest17320-21/usr/sbin/munin00:20:133
2876499241,22cyclictest14684-21apt21:25:133
2876499241,22cyclictest14660-21apache_accesses23:40:123
2876499241,22cyclictest10023-21irqstats00:15:183
2876499241,21cyclictest21343-21idleruntime-cro21:10:013
2876499241,21cyclictest21343-21idleruntime-cro21:10:003
2876499240,23cyclictest7683-21cpuspeed_turbos21:40:153
2876499240,23cyclictest6746-21chrt23:54:563
2876499240,23cyclictest30461-21cpu21:15:153
2876499240,23cyclictest27037-21grep00:25:213
2876499240,23cyclictest20786-21aten2_r7power_c23:05:143
2876499240,23cyclictest11000-21apache222:55:123
2876499240,21cyclictest141rcu_preempt23:46:133
2876499240,21cyclictest141rcu_preempt22:44:543
2876499240,21cyclictest141rcu_preempt00:02:183
2876499240,21cyclictest0-21swapper/321:33:573
2876199242,21cyclictest10624-21idleruntime22:20:172
2875899241,22cyclictest2393-21/usr/sbin/munin22:35:171
2875899241,22cyclictest1358-21apache_processe22:15:131
2875899240,21cyclictest141rcu_preempt22:11:011
2875899240,21cyclictest0-21swapper/123:45:191
2875899220,1cyclictest0-21swapper/120:20:531
2875499217,7cyclictest18013-21awk19:54:590
28754992121,0cyclictest0-21swapper/019:25:120
28758991914,4cyclictest12324-21tr20:50:151
28761991614,1cyclictest9391-21idleruntime-cro20:44:592
2875499166,5cyclictest225-21systemd-journal20:45:000
28764991510,3cyclictest19366-21cut19:55:193
2875899156,7cyclictest0-21swapper/120:25:161
2875499155,7cyclictest11012-21apache220:35:100
2875499154,6cyclictest18987-21munin-run21:05:010
2876499140,11cyclictest9659-21apt-get20:45:113
2876199146,7cyclictest0-21swapper/220:00:202
28761991413,1cyclictest32-21ksoftirqd/219:53:302
28761991410,4cyclictest0-21swapper/219:10:192
2875499142,10cyclictest27519-21idleruntime-cro20:14:590
2876499139,3cyclictest3772-21wc20:30:233
28764991310,2cyclictest23630-21grep20:05:163
28764991310,2cyclictest21753-21grep20:00:203
2876199138,3cyclictest0-21swapper/220:10:182
28761991310,2cyclictest9460-21cat19:35:192
28761991310,2cyclictest2741-21sensors_temp19:20:242
28761991310,2cyclictest2618-21grep20:30:152
2876199130,1cyclictest0-21swapper/220:25:232
2875899139,3cyclictest13747-21tr19:45:141
2875899139,3cyclictest11863-21tail19:40:171
2875499139,3cyclictest23101-21grep20:05:140
2875499139,2cyclictest19030-21cut19:55:170
2875499134,6cyclictest20659-21apt-get20:00:120
28754991310,2cyclictest13508-21wc20:50:250
2876499129,2cyclictest30895-21head20:20:173
28764991210,1cyclictest0-21swapper/320:50:133
2876199125,6cyclictest0-21swapper/219:55:132
2876199124,4cyclictest225-21systemd-journal20:50:002
2875899129,2cyclictest20724-21cut20:00:141
2875899129,2cyclictest11376-21tail20:45:241
2875899128,3cyclictest6984-21chrt20:39:591
2875499129,2cyclictest32700-21grep20:25:170
2875499129,2cyclictest15139-21sed19:45:270
2875499128,3cyclictest16085-21grep20:55:250
2875499123,4cyclictest225-21systemd-journal19:25:010
2876499118,2cyclictest891-21grep20:25:193
2876499118,2cyclictest30672-21tail19:10:323
2876499118,2cyclictest26609-21cut20:10:213
2876499117,3cyclictest6164-21grep19:30:133
2876199118,2cyclictest7202-21cat20:40:012
2876199118,2cyclictest31086-21cat20:20:182
2876199118,2cyclictest27918-21cut20:15:132
2876199118,2cyclictest15229-21grep19:45:262
2876199118,2cyclictest12371-21tr20:50:152
2876199118,2cyclictest11855-21grep19:40:172
2875899118,2cyclictest8313-21wc19:35:011
2875899118,2cyclictest3004-21grep19:20:271
2875899118,2cyclictest28512-21tail20:15:161
2875899118,2cyclictest21075-21taskset21:06:421
2875899118,2cyclictest21075-21taskset21:06:411
2875899118,2cyclictest20019-21chrt19:55:401
2875899118,2cyclictest10147-21tail19:35:261
2875899117,2cyclictest24650-21sed20:05:261
2875499118,2cyclictest9212-21tail19:35:170
2875499118,2cyclictest6721-21sort19:30:160
2875499118,2cyclictest12744-21sed19:40:260
2875499110,3cyclictest225-21systemd-journal20:30:000
2875499110,10cyclictest30992-21sed20:20:160
2876499108,1cyclictest8737-21cat20:40:233
2876499108,1cyclictest5952-21cat19:30:013
2876499108,1cyclictest31684-21expr19:15:153
2876499108,1cyclictest15848-21chrt19:50:093
2876499108,1cyclictest155712sleep319:49:313
2876499108,1cyclictest1439-21aten2_r7power_p19:20:143
2876499107,2cyclictest5029-21grep20:35:123
2876499107,2cyclictest27771-21basename20:15:103
2876499107,2cyclictest18291-21cat21:00:253
2876499107,2cyclictest16752-21date21:00:013
2876499107,2cyclictest11364-21grep19:40:143
2876499107,2cyclictest10086-21sed19:35:273
2876199107,2cyclictest7162-21cat19:30:192
2876199107,2cyclictest4655-21uname19:25:192
2876199107,2cyclictest23787-21sed20:05:172
2876199107,2cyclictest14602-21tr20:55:152
2876199101,6cyclictest32-21ksoftirqd/221:00:022
2875899108,1cyclictest9982sleep119:19:291
2875899108,1cyclictest5526-21chrt19:26:141
2875899108,1cyclictest17835-21taskset19:53:061
2875899107,2cyclictest7828-21sed20:40:141
2875899107,2cyclictest18732-21chrt21:02:031
2875899100,9cyclictest4601-21chrt20:34:531
2875899100,9cyclictest0-21swapper/119:10:191
2875499108,1cyclictest32092-21expr19:15:170
2875499108,1cyclictest29741-21chrt20:18:060
2875499107,2cyclictest30131-21grep19:10:270
2875499100,4cyclictest9691-21cut20:45:100
287619996,2cyclictest0-21swapper/221:05:152
287619996,2cyclictest0-21swapper/221:05:142
287619991,5cyclictest32-21ksoftirqd/219:18:442
287589996,2cyclictest25541-21cut20:10:141
287589996,2cyclictest15063-21grep20:55:161
287549997,1cyclictest20779-21wc21:05:280
287549997,1cyclictest20779-21wc21:05:270
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional