You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-17 - 20:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Tue Mar 17, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2585521190,0sleep00-21swapper/010:22:230
769829380,9sleep00-21swapper/007:05:330
591528874,10sleep20-21swapper/207:05:112
799828168,9sleep30-21swapper/307:09:263
800127858,16sleep10-21swapper/107:09:271
260352650,0sleep30-21swapper/310:41:383
254322570,0sleep20-21swapper/209:24:432
8132993113,17cyclictest32-21ksoftirqd/211:38:592
813099280,27cyclictest0-21swapper/110:23:461
813099260,23cyclictest32177-21pmu-power09:05:221
8132992520,1cyclictest32-21ksoftirqd/212:19:592
813099240,1cyclictest0-21swapper/110:00:111
8132992321,1cyclictest0-21swapper/210:10:122
8132992320,2cyclictest32-21ksoftirqd/209:45:162
812599230,1cyclictest0-21swapper/008:20:120
813599222,19cyclictest18000-21kworker/3:107:26:193
813599220,20cyclictest18000-21kworker/3:107:32:023
813599220,20cyclictest18000-21kworker/3:107:20:263
812599220,2cyclictest0-21swapper/010:25:150
813599210,13cyclictest29748-21runrttasks09:26:333
8132992116,4cyclictest32-21ksoftirqd/209:28:062
813299210,21cyclictest0-21swapper/209:35:382
8130992119,2cyclictest0-21swapper/112:34:471
813099210,3cyclictest0-21swapper/111:27:011
813099210,21cyclictest0-21swapper/109:24:271
813099210,1cyclictest644-21diskmemload12:14:551
813599200,19cyclictest0-21swapper/307:38:353
813299201,2cyclictest0-21swapper/209:10:122
8130992020,0cyclictest0-21swapper/109:51:451
813099200,20cyclictest0-21swapper/112:26:401
812599200,17cyclictest0-21swapper/011:43:420
813599190,2cyclictest0-21swapper/309:59:193
813599190,2cyclictest0-21swapper/309:59:183
8132991918,1cyclictest0-21swapper/212:06:282
813299190,18cyclictest0-21swapper/211:26:302
813099191,9cyclictest141rcu_preempt11:00:181
8130991917,1cyclictest0-21swapper/110:39:051
813099190,1cyclictest0-21swapper/110:05:131
8125991917,2cyclictest0-21swapper/012:13:530
8125991912,6cyclictest1790-21apache211:00:390
812599190,16cyclictest0-21swapper/009:56:200
812599190,16cyclictest0-21swapper/009:56:200
813599181,17cyclictest0-21swapper/311:06:233
813599180,17cyclictest1956-21kworker/3:210:25:033
8132991815,2cyclictest32-21ksoftirqd/210:23:582
813299181,1cyclictest0-21swapper/209:15:502
8130991815,2cyclictest55850irq/129-enp2s0-10:15:431
8130991815,2cyclictest0-21swapper/108:05:231
8130991815,2cyclictest0-21swapper/108:05:221
812599189,2cyclictest4904-21cut12:25:170
8125991815,1cyclictest0-21swapper/009:44:020
813599170,17cyclictest1956-21kworker/3:210:07:043
813299175,2cyclictest141rcu_preempt12:28:002
8132991711,4cyclictest32-21ksoftirqd/210:30:162
813299170,1cyclictest0-21swapper/212:24:082
8130991714,2cyclictest25-21ksoftirqd/109:35:371
8130991712,3cyclictest0-21swapper/109:14:071
812599179,2cyclictest32091-21tr09:05:210
812599171,6cyclictest17444-21munin-run09:20:010
8125991714,2cyclictest8116-21cyclictest11:12:000
812599170,13cyclictest26076-21ssh09:25:030
8135991612,2cyclictest6863-21grep10:50:133
813299161,5cyclictest32-21ksoftirqd/210:42:052
8132991615,1cyclictest0-21swapper/210:54:022
8132991613,2cyclictest311rcuc/212:35:252
813299161,1cyclictest141rcu_preempt12:02:232
813299160,15cyclictest0-21swapper/211:50:272
8130991615,1cyclictest38150irq/131-snd_hda10:13:301
8130991614,1cyclictest0-21swapper/110:55:141
813099160,2cyclictest644-21diskmemload11:35:211
813099160,15cyclictest0-21swapper/111:52:011
813099160,15cyclictest0-21swapper/111:30:381
8125991610,2cyclictest8283-21tr08:15:200
8135991515,0cyclictest0-21swapper/311:29:003
8135991514,1cyclictest0-21swapper/312:26:383
813599150,2cyclictest0-21swapper/309:51:003
813299158,6cyclictest2170-21/usr/sbin/munin11:45:162
813299158,3cyclictest32-21ksoftirqd/211:00:472
813299153,2cyclictest32-21ksoftirqd/210:26:022
813299153,11cyclictest32-21ksoftirqd/210:36:382
8132991512,2cyclictest311rcuc/211:30:142
8132991510,2cyclictest32-21ksoftirqd/209:40:172
8130991513,2cyclictest38150irq/131-snd_hda12:15:181
8130991512,2cyclictest25-21ksoftirqd/111:20:001
8125991511,3cyclictest11490-21sh11:30:380
813599142,9cyclictest0-21swapper/312:10:053
8135991411,2cyclictest25445-21grep08:50:243
8135991410,3cyclictest22497-21ls10:20:183
813299149,2cyclictest32-21ksoftirqd/211:19:412
813299143,7cyclictest28935-21/usr/sbin/munin10:05:142
8132991411,2cyclictest311rcuc/211:40:142
8132991410,4cyclictest32-21ksoftirqd/211:22:382
8132991410,4cyclictest32-21ksoftirqd/211:22:382
813299140,1cyclictest0-21swapper/211:56:282
813299140,13cyclictest13540-21grep08:25:272
813099148,3cyclictest38150irq/131-snd_hda11:08:101
8130991412,1cyclictest644-21diskmemload12:38:111
8130991412,1cyclictest25-21ksoftirqd/109:29:211
8130991410,3cyclictest4646-21wc12:05:231
8130991410,3cyclictest2160-21wc08:05:001
813099140,14cyclictest0-21swapper/110:53:051
813099140,14cyclictest0-21swapper/110:46:371
812599148,2cyclictest22103-21chrt07:36:470
8125991410,4cyclictest304582sleep011:23:160
8125991410,4cyclictest304582sleep011:23:150
8125991410,3cyclictest29339-21wc12:20:190
8125991410,3cyclictest28883-21tr10:05:140
8125991410,3cyclictest12452-21ssh10:15:010
812599140,12cyclictest21703-21tr08:45:150
812599140,12cyclictest0-21swapper/012:05:140
813599139,3cyclictest537-21snmpd10:02:583
813599139,3cyclictest25651-21tail11:20:173
813599139,3cyclictest25651-21tail11:20:163
813599139,2cyclictest26256-21cut08:55:163
8135991310,2cyclictest8774-21sed08:15:273
8135991310,2cyclictest5796-21grep10:10:183
8135991310,2cyclictest3828-21cat12:25:003
8135991310,2cyclictest28183-21grep09:45:193
813299132,6cyclictest644-21diskmemload11:13:572
8132991311,2cyclictest32-21ksoftirqd/209:54:422
8132991310,2cyclictest29152-21cat07:50:242
813299130,2cyclictest9446-21kworker/u8:110:57:452
813299130,1cyclictest0-21swapper/210:00:112
813299130,12cyclictest31343-21cat07:55:222
813099139,3cyclictest16592-21cat08:35:011
813099135,2cyclictest225-21systemd-journal09:55:221
813099135,2cyclictest225-21systemd-journal09:55:221
813099131,10cyclictest141rcu_preempt11:47:321
8130991310,2cyclictest9110-21grep07:10:191
8130991310,2cyclictest25-21ksoftirqd/109:35:001
812599139,3cyclictest2957-21cut11:45:170
812599139,2cyclictest32098-21uname10:45:190
812599135,4cyclictest225-21systemd-journal12:00:000
812599133,4cyclictest644-21diskmemload11:25:130
8125991310,2cyclictest31598-21grep07:55:250
8125991310,2cyclictest15238-21cron07:25:000
8125991310,2cyclictest12385-21grep07:15:260
813599129,2cyclictest9977-21grep07:10:283
813599129,2cyclictest9378-21ssh12:08:443
813599129,2cyclictest8635-21sed11:10:143
813599129,2cyclictest6575-21sed10:30:173
813599129,2cyclictest32636-21tr08:00:143
813599129,2cyclictest26916-21grep11:40:183
813599129,2cyclictest23387-21grep07:40:183
813599129,2cyclictest22146-21ssh12:35:193
813599129,2cyclictest21814-21sed12:15:253
813599129,2cyclictest10608-21apache_volume09:35:123
813599121,2cyclictest225-21systemd-journal09:25:003
813599120,1cyclictest0-21swapper/312:04:243
813299129,2cyclictest31444-21sh10:45:152
813299129,2cyclictest22216-21tail08:45:172
813299129,2cyclictest15338-21taskset12:31:042
813299129,2cyclictest13436-21grep12:10:262
813299129,2cyclictest13220-21sh10:15:142
813299129,2cyclictest107472sleep207:15:132
813299120,11cyclictest17346-21sort08:35:162
813099129,2cyclictest6874-21cron08:15:001
813099129,2cyclictest26722-21sendmail-msp12:00:001
813099129,2cyclictest20529-21grep07:35:141
813099129,2cyclictest14024-21cron08:30:001
813099127,4cyclictest9430-21ssh09:15:071
813099127,4cyclictest27280-21ssh11:20:461
813099127,4cyclictest27280-21ssh11:20:451
813099120,9cyclictest0-21swapper/107:50:121
813099120,1cyclictest0-21swapper/110:42:461
813099120,1cyclictest0-21swapper/109:45:371
812599129,2cyclictest5011-21tr08:10:150
812599129,2cyclictest29840-21wc09:00:220
812599129,2cyclictest24759-21sh07:45:000
812599129,2cyclictest15902-21ssh10:55:150
812599129,2cyclictest15614-21grep10:35:220
812599129,2cyclictest14898-21tail10:15:250
812599129,2cyclictest14448-21sed12:30:270
812599128,3cyclictest21693-21tail12:35:170
812599128,3cyclictest10229-21wc09:35:000
812599128,2cyclictest27361-21cat07:50:010
812599120,12cyclictest0-21swapper/012:15:230
812599120,11cyclictest27508-21cut09:45:150
812599120,11cyclictest0-21swapper/009:10:230
813599119,1cyclictest19703-21sed11:35:253
813599118,2cyclictest6790-21ssh11:47:253
813599118,2cyclictest32733-21chrt09:05:563
813599118,2cyclictest30242-21ssh11:03:563
813599118,2cyclictest2611-21tr09:30:143
813599118,2cyclictest2069-21grep09:10:163
813599118,2cyclictest20687-21awk09:40:253
813599118,2cyclictest1978-21ssh10:46:353
813599118,2cyclictest19363-21ssh11:16:073
813599118,2cyclictest17890-21ssh10:17:433
813599118,2cyclictest17874-21sh10:36:483
813599118,2cyclictest17069-21ssh10:55:273
813599118,2cyclictest13276-21sort12:30:173
813599118,2cyclictest12291-21ssh09:15:593
813599118,2cyclictest10652-21ssh11:30:193
813599118,2cyclictest10385-21ssh11:50:033
813599117,3cyclictest12158-21wc07:15:253
813599111,5cyclictest225-21systemd-journal08:25:003
813599110,1cyclictest0-21swapper/311:55:143
813299119,1cyclictest3126-21grep08:05:162
813299119,1cyclictest3126-21grep08:05:162
813299118,2cyclictest32306-21grep09:05:222
813299118,2cyclictest25729-21grep07:45:172
813299118,2cyclictest21493-21head07:35:212
813299118,2cyclictest20594-21cat08:40:262
813299118,2cyclictest19816-21taskset07:32:542
813299118,2cyclictest14347-21ssh09:55:502
813299118,2cyclictest14347-21ssh09:55:492
813299115,4cyclictest311rcuc/209:31:542
813299112,3cyclictest1678-21kworker/u8:111:07:102
813099118,3cyclictest21828-21ssh09:41:071
813099118,2cyclictest7895-21ssh11:10:051
813099118,2cyclictest27188-21cat08:55:221
813099118,2cyclictest2539-21ssh12:24:081
813099118,2cyclictest25290-21tr07:45:151
813099118,2cyclictest24013-21cat08:50:111
813099118,2cyclictest11657-21idleruntime-cro08:25:001
813099117,3cyclictest5327-21ls10:29:591
813099117,3cyclictest29123-21df09:00:151
813099117,1cyclictest141rcu_preempt11:40:131
813099112,9cyclictest0-21swapper/110:30:261
813099110,1cyclictest0-21swapper/112:03:371
812599118,2cyclictest8797-21expr07:10:170
812599118,2cyclictest8284-21ssh10:50:220
812599118,2cyclictest27568-21ssh10:04:360
812599118,2cyclictest2636-21tr08:05:150
812599118,2cyclictest2636-21tr08:05:140
812599118,2cyclictest24115-21ssh11:38:470
812599118,2cyclictest22818-21ssh09:22:400
812599118,2cyclictest22314-21ssh10:40:050
812599118,2cyclictest1986-21ssh11:05:360
812599118,2cyclictest16216-21ssh11:15:010
812599118,2cyclictest12162-21tr08:25:160
812599118,2cyclictest11336-21uniq09:35:160
812599118,2cyclictest10326-21ssh11:50:010
812599117,3cyclictest28033-21ssh12:00:170
812599117,3cyclictest27678-21grep07:50:120
812599117,3cyclictest18534-21cut07:30:170
812599112,4cyclictest0-21swapper/009:50:140
813599108,1cyclictest65302sleep308:10:393
813599108,1cyclictest29484-21chrt07:52:083
813599108,1cyclictest21884-21expr08:45:153
813599107,2cyclictest29981-21apache_accesses07:55:123
813599107,2cyclictest29357-21head09:00:173
813599107,2cyclictest25154-21basename07:45:133
813599107,2cyclictest19176-21aten2_r7power_v08:40:143
813599107,2cyclictest16905-21tr08:35:133
813599107,2cyclictest15073-21grep08:30:183
813299108,2cyclictest311rcuc/207:40:112
813299108,1cyclictest981-21cat08:00:192
813299108,1cyclictest7567-21expr08:15:162
813299108,1cyclictest30108-21cat09:00:262
813299108,1cyclictest25672-21taskset08:51:592
813299108,1cyclictest15717-21cat07:25:152
813299107,2cyclictest9619-21cut08:20:122
813299107,2cyclictest32-21ksoftirqd/208:10:212
813299107,2cyclictest14298-21apache_volume08:30:122
813299107,2cyclictest13337-21cat07:20:152
813299107,2cyclictest10150-21taskset07:11:032
813099108,1cyclictest22218-21expr08:45:171
813099108,1cyclictest196542sleep107:30:461
813099108,1cyclictest18718-21chrt08:39:221
813099107,2cyclictest29960-21cat07:55:111
813099107,2cyclictest24399-21taskset07:40:301
813099107,2cyclictest11778-21cat07:15:191
813099102,4cyclictest225-21systemd-journal08:18:451
813099100,9cyclictest190222sleep108:40:031
812599108,1cyclictest54592sleep010:30:010
812599108,1cyclictest25003-21cut08:50:190
812599108,1cyclictest16495-21wc07:25:200
812599107,2cyclictest875-21cut08:00:190
812599107,2cyclictest26421-21tr08:55:170
812599107,2cyclictest19021-21taskset08:40:010
812599107,2cyclictest18086-21wc08:35:240
812599107,2cyclictest14319-21aten2_r7power_c08:30:150
81359996,2cyclictest2430-21grep08:05:143
81359996,2cyclictest2430-21grep08:05:133
81329990,1cyclictest0-21swapper/208:55:152
81309997,1cyclictest14999-21taskset07:22:011
81309990,5cyclictest225-21systemd-journal07:30:001
81359986,1cyclictest12009-21cut08:25:163
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional