You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-15 - 20:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Sat Aug 15, 2026 12:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2342522980,4sleep339-21rcuc/307:05:173
245982237162,24sleep10-21swapper/107:09:291
246362201164,25sleep00-21swapper/007:09:570
243552201165,24sleep20-21swapper/207:06:202
1821021850,4sleep12472699cyclictest09:20:121
2534621710,4sleep22473399cyclictest08:20:262
2473399108104,3cyclictest0-21swapper/210:54:252
24733998480,3cyclictest0-21swapper/209:19:312
250962560,4sleep10-21swapper/110:50:021
2473399411,28cyclictest16477-21/usr/sbin/munin08:00:222
2473399381,18cyclictest0-21swapper/209:49:012
24733993726,3cyclictest0-21swapper/212:06:532
24733993625,3cyclictest0-21swapper/209:07:212
24733993625,3cyclictest0-21swapper/209:07:212
2472199367,13cyclictest22236-21awk09:30:000
2472199358,13cyclictest11847-21munin-run10:20:010
2472199358,13cyclictest11847-21munin-run10:20:010
2472199356,13cyclictest18460-21awk10:35:010
24721993510,12cyclictest10620-21munin-run07:50:010
24733993422,7cyclictest0-21swapper/209:55:172
2473399341,14cyclictest0-21swapper/211:55:452
24733993319,9cyclictest0-21swapper/211:12:092
2473399331,20cyclictest0-21swapper/207:46:232
2473399321,19cyclictest0-21swapper/208:06:042
2473399321,15cyclictest0-21swapper/212:12:572
2473399321,13cyclictest0-21swapper/210:06:082
2473399320,16cyclictest0-21swapper/210:42:012
24721993211,17cyclictest23807-21munin-run08:20:010
24721993211,17cyclictest16247-21munin-run10:30:010
24733993117,4cyclictest0-21swapper/210:55:212
2473399311,21cyclictest0-21swapper/211:27:492
2473399311,21cyclictest0-21swapper/211:27:492
2473399311,21cyclictest0-21swapper/210:19:522
2473399311,21cyclictest0-21swapper/210:19:522
2473399311,19cyclictest0-21swapper/209:23:412
2472199311,14cyclictest20034-21munin-run09:25:000
2473399308,10cyclictest827-21runrttasks09:02:332
2473399301,21cyclictest0-21swapper/207:59:522
2473399301,21cyclictest0-21swapper/207:59:522
2473399301,14cyclictest0-21swapper/210:02:322
2473399301,13cyclictest0-21swapper/209:44:412
2473399301,12cyclictest0-21swapper/211:39:092
2473399301,12cyclictest0-21swapper/211:15:332
2473399300,14cyclictest0-21swapper/209:31:332
2472199302,13cyclictest19425-21idleruntime-cro08:10:010
2472199301,14cyclictest4635-21munin-run08:50:010
24733992912,4cyclictest17914-21kernelversion11:45:192
2473399291,19cyclictest0-21swapper/207:23:232
2473399291,14cyclictest0-21swapper/212:25:532
2473399291,13cyclictest0-21swapper/207:38:072
2473399291,10cyclictest0-21swapper/212:16:132
2473399290,15cyclictest0-21swapper/211:00:412
2473399290,13cyclictest0-21swapper/208:41:372
24733992812,3cyclictest0-21swapper/208:30:052
2473399281,14cyclictest0-21swapper/212:02:532
2473399281,13cyclictest0-21swapper/210:23:042
2473399281,12cyclictest0-21swapper/207:10:392
2473399281,11cyclictest0-21swapper/210:26:532
2473399281,11cyclictest0-21swapper/208:28:072
2473399281,10cyclictest2102-21latency_hist12:25:012
24733992810,3cyclictest0-21swapper/209:35:532
2472699281,19cyclictest0-21swapper/109:58:481
2472199282,12cyclictest8414-21munin-run07:45:010
2472199280,12cyclictest13428-21idleruntime-cro09:10:000
2472199280,12cyclictest13428-21idleruntime-cro09:10:000
24733992718,3cyclictest0-21swapper/208:57:532
2473399271,13cyclictest0-21swapper/210:47:132
2473399271,12cyclictest0-21swapper/212:37:082
2473399270,9cyclictest31988-21timerandwakeup08:35:252
2473399270,15cyclictest0-21swapper/211:40:132
2473399270,14cyclictest0-21swapper/210:10:282
24733992618,3cyclictest0-21swapper/208:13:562
24733992614,3cyclictest0-21swapper/207:33:432
24733992612,3cyclictest0-21swapper/211:09:012
2473399261,13cyclictest5016-21kworker/u8:011:23:012
2473399261,13cyclictest0-21swapper/212:30:452
2473399261,12cyclictest0-21swapper/210:35:572
2473399261,10cyclictest0-21swapper/209:52:052
2473399260,12cyclictest0-21swapper/207:25:192
2472699261,16cyclictest5603-21tr10:05:131
2473899251,15cyclictest87350irq/126-enp1s0-10:40:123
2473899251,11cyclictest0-21swapper/310:23:173
2473399251,13cyclictest0-21swapper/211:51:332
2473399251,13cyclictest0-21swapper/210:31:052
2473399251,10cyclictest3577-21memory08:45:192
2473399251,10cyclictest0-21swapper/209:14:192
2473399251,10cyclictest0-21swapper/208:15:572
24733992510,3cyclictest0-21swapper/208:50:132
2472699253,10cyclictest0-21swapper/110:20:191
24726992510,9cyclictest0-21swapper/110:39:071
2472199259,14cyclictest1834-21perf07:30:010
2472199251,13cyclictest28819-21munin-run09:45:020
2473399249,3cyclictest0-21swapper/207:18:312
2473399241,12cyclictest0-21swapper/207:51:262
2473399240,11cyclictest0-21swapper/211:31:132
2472699241,12cyclictest19982-21chrt11:50:171
24726992411,11cyclictest10561-21kworker/u8:007:45:051
2472199245,3cyclictest87550irq/128-enp1s0-12:35:260
2473899230,11cyclictest0-21swapper/311:38:493
2473899230,10cyclictest0-21swapper/311:59:433
2473399231,10cyclictest0-21swapper/209:26:332
2473399230,21cyclictest0-21swapper/207:43:072
24726992317,3cyclictest15904-21hddtemp_smartct08:00:161
2472699231,19cyclictest26385-21pmu-power10:50:241
2472699231,19cyclictest12534-21ntp_states09:05:211
2472699231,19cyclictest12534-21ntp_states09:05:211
2472699231,14cyclictest0-21swapper/111:27:311
2472699231,14cyclictest0-21swapper/111:27:311
24721992310,2cyclictest7138-21kworker/0:210:36:530
27032220,1sleep10-21swapper/107:30:151
2473899227,7cyclictest0-21swapper/310:10:223
2473899221,15cyclictest0-21swapper/311:34:423
24738992211,3cyclictest0-21swapper/309:54:133
2473899221,10cyclictest0-21swapper/309:42:413
2472699229,11cyclictest2776-21kworker/u8:107:29:571
2472699221,11cyclictest525-21tr08:40:131
2472699221,11cyclictest17910-21irqstats11:45:191
2472699221,10cyclictest26346-21kworker/u8:009:52:091
2472699220,17cyclictest0-21swapper/110:32:191
2472199223,2cyclictest24282-21kworker/0:109:35:230
2472199221,7cyclictest0-21swapper/011:52:310
2472199221,6cyclictest0-21swapper/010:40:550
2473899218,4cyclictest19080-21awk11:50:013
2473899215,2cyclictest87350irq/126-enp1s0-11:14:523
2473899211,4cyclictest0-21swapper/309:34:593
2473899211,3cyclictest0-21swapper/312:23:153
2473899211,15cyclictest41-21ksoftirqd/310:46:123
2473899211,13cyclictest0-21swapper/311:26:273
2473899211,13cyclictest0-21swapper/311:26:273
2473899211,12cyclictest839-21lldpd10:35:013
2473899211,12cyclictest0-21swapper/309:25:443
2473899211,11cyclictest30243-21perl11:00:153
2473899211,11cyclictest1399-21df09:55:143
2473899210,6cyclictest10504-21kworker/u8:209:35:063
2472699219,2cyclictest87650irq/129-enp1s0-10:25:191
2472699217,4cyclictest22554-21ntp_states11:55:201
2472699211,18cyclictest0-21swapper/112:05:241
2472699211,18cyclictest0-21swapper/108:15:291
2472699211,14cyclictest0-21swapper/109:38:401
2472699211,12cyclictest0-21swapper/111:40:311
2472699211,11cyclictest29056-21missed_timers12:10:201
2472699211,11cyclictest25626-21users08:20:271
2472699211,11cyclictest11767-21smart_sda11:30:251
2472699211,10cyclictest0-21swapper/111:21:001
2472699210,15cyclictest0-21swapper/110:41:161
2472199211,10cyclictest0-21swapper/009:13:220
96342201,2sleep00-21swapper/010:14:580
2473899208,4cyclictest0-21swapper/312:03:063
2473899208,4cyclictest0-21swapper/307:35:273
2473899205,8cyclictest87350irq/126-enp1s0-07:55:193
2473899205,8cyclictest87350irq/126-enp1s0-07:55:193
2473899203,9cyclictest0-21swapper/312:35:153
2473899201,15cyclictest13433-21diskmemload11:25:013
2473899201,13cyclictest0-21swapper/310:33:173
2473899201,11cyclictest30631-21/usr/sbin/munin08:35:163
2473899201,11cyclictest14899-21/usr/sbin/munin11:40:223
2473899201,10cyclictest30400-21latency_hist08:35:013
2473899201,10cyclictest10481-21tail10:15:163
2473899201,10cyclictest10481-21tail10:15:163
2473899200,15cyclictest24712-21cyclictest11:51:373
2472699208,4cyclictest0-21swapper/111:07:121
2472699208,4cyclictest0-21swapper/109:25:401
2472699207,3cyclictest0-21swapper/108:37:181
2472699205,12cyclictest6504-21cpuspeed_turbos07:40:111
2472699201,7cyclictest29605-21ps08:30:231
24726992016,3cyclictest0-21swapper/111:01:271
2472699201,2cyclictest0-21swapper/107:39:071
24726992012,5cyclictest6825-21dump-pmu-power08:55:011
2472699201,18cyclictest0-21swapper/110:15:231
2472699201,18cyclictest0-21swapper/110:15:231
2472699201,18cyclictest0-21swapper/110:11:461
2472699201,17cyclictest0-21swapper/111:16:271
2472699201,14cyclictest0-21swapper/107:11:111
2472699201,11cyclictest987-21latency12:20:191
2472699201,11cyclictest27883-21cpuspeed_turbos07:15:131
2472699201,11cyclictest0-21swapper/109:41:131
2472699201,10cyclictest579-21systemd-journal09:35:001
2472699201,10cyclictest0-21swapper/107:50:191
2472699200,7cyclictest0-21swapper/112:37:031
2472699200,4cyclictest0-21swapper/112:17:581
2472699200,18cyclictest0-21swapper/107:22:141
2472199209,3cyclictest25271-21/usr/sbin/munin10:50:230
2472199203,15cyclictest7-21ksoftirqd/009:20:000
24721992014,3cyclictest13650-21grep10:20:270
24721992013,4cyclictest3920-21grep10:00:190
2472199201,13cyclictest0-21swapper/011:38:300
2472199201,10cyclictest2333-21/usr/sbin/munin12:25:180
292572190,5sleep011-21rcuc/008:30:190
282582190,3sleep30-21swapper/308:30:013
2473899197,3cyclictest10742-21latency_hist07:50:013
2473899192,4cyclictest6831-21munin-run08:55:013
2473899191,9cyclictest0-21swapper/310:29:063
2473899191,7cyclictest27252-21ls12:05:243
2473899191,6cyclictest28531-21ntp_states10:55:213
2473899191,5cyclictest13812-21grep09:10:133
2473899191,12cyclictest0-21swapper/312:18:233
2473899191,11cyclictest0-21swapper/309:15:163
2473899190,5cyclictest0-21swapper/310:00:163
2473899190,11cyclictest0-21swapper/309:21:023
2472699198,2cyclictest20626-21grep08:10:201
2472699197,3cyclictest17121-21smart_sda09:15:231
2472699193,9cyclictest14356-21kworker/u8:212:26:051
2472699192,10cyclictest0-21swapper/112:04:151
2472699191,9cyclictest2702-21cpu08:45:131
2472699191,9cyclictest13433-21diskmemload10:55:131
24726991915,3cyclictest0-21swapper/109:01:491
24726991912,4cyclictest8366-21grep08:55:251
2472699191,16cyclictest0-21swapper/111:10:231
2472699191,16cyclictest0-21swapper/108:05:361
2472699191,11cyclictest0-21swapper/110:00:231
2472699191,10cyclictest13433-21diskmemload12:30:061
2472699190,1cyclictest0-21swapper/108:27:421
2472699190,18cyclictest0-21swapper/107:57:321
2472699190,18cyclictest0-21swapper/107:57:321
2472199196,5cyclictest87350irq/126-enp1s0-08:25:010
2472199194,9cyclictest0-21swapper/007:55:140
2472199194,9cyclictest0-21swapper/007:55:140
2472199191,5cyclictest0-21swapper/009:55:270
24721991913,3cyclictest31405-21ls12:15:230
2472199191,12cyclictest0-21swapper/010:49:170
2472199191,10cyclictest6293-21irqstats10:05:180
2472199191,10cyclictest23691-21/usr/sbin/munin12:00:150
2473899187,8cyclictest0-21swapper/307:20:163
2473899187,3cyclictest0-21swapper/309:47:193
2473899184,3cyclictest14356-21kworker/u8:212:28:413
2473899182,4cyclictest0-21swapper/312:10:243
2473899182,13cyclictest41-21ksoftirqd/312:30:133
2473899181,7cyclictest0-21swapper/310:09:423
2473899181,10cyclictest31871-21/usr/sbin/munin11:05:133
2473899181,10cyclictest0-21swapper/310:51:223
24726991814,3cyclictest0-21swapper/109:46:571
2472699181,16cyclictest0-21swapper/111:37:251
2472699181,15cyclictest0-21swapper/109:10:101
2472199189,6cyclictest7-21ksoftirqd/012:00:020
2472199183,12cyclictest357-21/usr/sbin/munin08:40:160
2472199182,7cyclictest24925-21/usr/sbin/munin07:10:130
2472199182,3cyclictest0-21swapper/012:14:550
2472199182,3cyclictest0-21swapper/007:16:190
2472199181,9cyclictest808-21snmpd11:13:210
2472199181,6cyclictest9373-21ntp_states11:25:200
2472199181,6cyclictest9373-21ntp_states11:25:200
2472199181,6cyclictest0-21swapper/009:45:170
2472199181,6cyclictest0-21swapper/009:30:570
2472199181,3cyclictest10854-21diskstats11:30:140
2472199181,14cyclictest6499-21ls12:35:010
159612181,2sleep00-21swapper/008:00:180
111272181,3sleep30-21swapper/309:03:463
37812171,2sleep30-21swapper/307:32:083
278422170,3sleep00-21swapper/008:25:300
2473899176,3cyclictest87350irq/126-enp1s0-07:10:143
2473899176,3cyclictest17441-21/usr/sbin/munin08:05:233
2473899176,2cyclictest4730-21kernelversion11:15:183
2473899173,2cyclictest87650irq/129-enp1s0-08:40:183
2473899172,12cyclictest39-21rcuc/308:15:223
2473899171,9cyclictest0-21swapper/307:28:043
2473899171,6cyclictest0-21swapper/309:05:163
2473899171,6cyclictest0-21swapper/309:05:163
2473899171,4cyclictest7779-21threads07:40:243
24738991712,3cyclictest21643-21cat08:15:023
24738991712,3cyclictest16821-21users08:00:263
2473899171,14cyclictest0-21swapper/308:48:453
24738991711,3cyclictest24500-21grep08:20:153
24738991710,4cyclictest0-21swapper/307:15:193
2473899170,15cyclictest0-21swapper/308:57:123
2472199176,3cyclictest87550irq/128-enp1s0-07:50:270
2472199172,12cyclictest9253-21/usr/sbin/munin09:00:110
2472199171,8cyclictest0-21swapper/011:01:500
2472199171,7cyclictest0-21swapper/011:16:160
2472199171,6cyclictest0-21swapper/012:23:410
2472199171,4cyclictest2047-21/usr/sbin/munin07:30:190
24721991713,3cyclictest7-21ksoftirqd/008:35:190
24721991712,3cyclictest32317-21sh09:50:220
24721991711,4cyclictest27470-21acpi10:55:120
24721991711,3cyclictest5183-21expr08:50:130
24721991710,4cyclictest30644-21sed07:20:160
24721991710,4cyclictest18149-21tr11:45:210
2472199170,2cyclictest81ktimersoftd/011:20:200
2472199170,14cyclictest964-21grep11:05:240
2473899161,5cyclictest0-21swapper/307:54:403
2472199162,5cyclictest4246-21/usr/sbin/munin07:35:170
2472199161,6cyclictest0-21swapper/012:05:150
2472199161,3cyclictest0-21swapper/008:58:560
24721991611,3cyclictest16995-21cut11:45:010
213372161,2sleep00-21swapper/008:11:420
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional