You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-11 - 18:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Sun Jan 11, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
94202366913,3643sleep027806-21kworker/0:210:31:250
161002138815,1359sleep131588-1kworker/1:2H09:10:361
7960212768,1254sleep37694-1kworker/3:1H10:27:413
14224212425,1211sleep131588-1kworker/1:2H12:35:081
7195210455,1030sleep131588-1kworker/1:2H12:11:141
1531926107,592sleep222605-1kworker/2:0H12:38:492
19983257713,549sleep020056-21users07:40:140
2476325557,539sleep222605-1kworker/2:0H11:22:262
1063825328,513sleep03953-1kworker/0:1H08:55:170
28798245337,400sleep128853-21find11:35:091
11235243611,417sleep37694-1kworker/3:1H10:36:453
988624228,405sleep03953-1kworker/0:1H08:53:150
18408234715,315sleep318511-21unin-run09:19:433
13740227521,226sleep013938-21latency_hist12:34:430
2474226315,123sleep10-21swapper/110:10:111
1136422587,240sleep1378-21sshd10:38:251
859122477,227sleep37694-1kworker/3:1H12:16:413
18276224556,78sleep118293-21modprobe09:19:411
4935223529,152sleep14938-21cron12:04:401
757221871,88sleep10-21swapper/110:05:061
27254221313,151sleep127337-21mailstats11:30:091
14223221331,129sleep10-21swapper/109:05:111
26479219158,120sleep126507-21latency_hist11:29:401
1308321899,81sleep113223-21unixbench_singl12:30:121
17763218615,155sleep317821-21cat10:59:423
17683218634,83sleep117686-21cron10:59:401
1342321768,153sleep113570-21cat09:04:431
1342321768,153sleep113570-21cat09:04:431
15097217321,98sleep115203-21ls07:25:131
1162821678,17sleep10-21swapper/107:15:081
17588216110,30sleep10-21swapper/107:35:021
28618215862,49sleep10-21swapper/109:50:091
473621486,11sleep00-21swapper/008:35:180
3119821489,83sleep10-21swapper/111:44:421
7170214521,13sleep10-21swapper/110:25:331
1996721458,124sleep37694-1kworker/3:1H09:23:063
1996721458,124sleep37694-1kworker/3:1H09:23:063
22339214424,105sleep122372-21fschecks_count07:50:001
3210021419,120sleep132163-21sendmail_mailqu08:20:131
21276214147,37sleep10-21swapper/107:45:121
2970821386,14sleep10-21swapper/109:54:571
1731121388,120sleep117341-21unin-run07:34:421
10306213810,75sleep110439-21latency_hist08:54:451
6485213731,56sleep16526-21http12:09:411
25063213222,85sleep025068-21modprobe07:59:400
2120221318,14sleep00-21swapper/007:45:110
9693213047,65sleep19705-21munin-run10:34:411
3141021267,104sleep131557-21cut08:19:441
3141021267,104sleep131557-21cut08:19:441
29974212614,98sleep229999-21grep11:39:592
17063212611,98sleep03953-1kworker/0:1H09:15:000
8428212110,55sleep18612-21latency_hist08:49:441
21587211911,89sleep121647-21find11:10:101
20459211910,51sleep10-21swapper/107:44:431
355621186,96sleep10-21swapper/110:14:591
314821177,98sleep00-21swapper/010:14:050
18003211713,43sleep19200-21cyclictest10:59:571
23498211614,78sleep223499-21192.168.115.60-07:54:292
10051211616,85sleep210139-21systemd-run08:54:422
16481211513,86sleep316492-21ntpq07:30:083
1990221145,93sleep10-21swapper/109:22:151
1990221145,93sleep10-21swapper/109:22:151
25841211322,36sleep10-21swapper/108:00:101
11751211210,88sleep311783-21/usr/sbin/munin07:15:103
500210817,71sleep0644-21cat08:24:440
1280121085,92sleep10-21swapper/110:41:361
2508210712,35sleep10-21swapper/111:55:071
2492321065,58sleep10-21swapper/107:57:561
9594210551,15sleep10-21swapper/107:10:071
9594210551,15sleep10-21swapper/107:10:071
12105210534,56sleep112212-21latency_hist12:29:431
5535210414,75sleep25558-21/usr/sbin/munin10:20:092
5535210414,75sleep25558-21/usr/sbin/munin10:20:092
1186121047,63sleep111868-21munin-run08:59:421
16191210323,59sleep20-21swapper/209:10:372
12938210342,21sleep112931-21apt-get07:19:561
2233221028,14sleep00-21swapper/007:49:590
13099210215,64sleep312912-21/usr/sbin/munin07:20:023
196921015,84sleep00-21swapper/008:29:320
13039210113,75sleep013086-21systemd-run10:44:420
32549210042,24sleep11-21systemd10:04:421
20245210039,23sleep120390-21sed11:05:131
2474229912,21sleep324771-21munin-run09:39:413
170029819,55sleep21825-21latency_hist10:09:432
2877829710,15sleep00-21swapper/011:35:080
391729634,16sleep10-21swapper/108:35:021
319932967,66sleep00-21swapper/010:00:130
275702965,81sleep10-21swapper/109:45:531
1492029632,42sleep214954-21munin-run09:09:402
58382957,31sleep10-21swapper/108:40:071
2978129512,19sleep00-21swapper/008:14:420
2978129512,19sleep00-21swapper/008:14:420
53829410,68sleep10-21swapper/111:49:551
26242940,11sleep30-21swapper/310:10:123
2119129414,13sleep30-21swapper/311:10:003
921729310,55sleep30-21swapper/312:19:483
3792938,16sleep00-21swapper/010:04:530
320442928,68sleep10-21swapper/111:45:101
12762927,13sleep10-21swapper/108:25:131
1126329219,26sleep00-21swapper/012:25:100
2348128922,52sleep10-21swapper/109:34:551
64712888,62sleep16609-21gpgv10:24:441
64712888,62sleep16609-21gpgv10:24:441
1520528814,60sleep215339-21unixbench_singl07:25:142
866628713,15sleep20-21swapper/210:30:022
65428712,60sleep3669-21cut10:05:043
300192860,8sleep30-21swapper/309:55:063
2594228625,31sleep30-21swapper/311:26:403
705128516,50sleep07000-21apt-get12:10:250
277928511,59sleep22863-21grep08:30:112
1913028412,25sleep30-21swapper/307:39:533
475328327,28sleep30-21swapper/310:19:413
44662839,11sleep00-21swapper/010:16:070
2669828311,56sleep326790-21latency_hist08:04:443
2464228317,49sleep10-21swapper/111:20:561
226552839,16sleep10-21swapper/111:14:541
1245928315,53sleep312580-21timerwakeupswit10:40:133
316222820,6sleep30-21swapper/310:00:033
2716228221,47sleep027232-21switchtime09:45:110
1880628225,37sleep018811-21sort09:19:520
1880628225,37sleep018811-21sort09:19:520
287522817,63sleep128802-21sed08:10:031
287522817,63sleep128802-21sed08:10:031
1942428121,43sleep20-21swapper/209:20:272
1942428121,43sleep20-21swapper/209:20:272
105562807,33sleep110653-21date12:24:431
27372798,58sleep12778-21cut08:30:101
229142795,62sleep10-21swapper/109:31:081
1953227917,49sleep019551-21apt-get11:04:520
1729227925,30sleep00-21swapper/007:34:410
2856927824,34sleep20-21swapper/208:10:002
2856927824,34sleep20-21swapper/208:10:002
273352788,35sleep127377-21ntpq08:05:061
2505927811,51sleep10-21swapper/109:39:561
281312777,17sleep00-21swapper/009:49:510
166862779,54sleep00-21swapper/009:14:150
670327612,19sleep00-21swapper/008:44:420
2169327615,48sleep121780-21latency_hist09:29:421
1528927610,11sleep00-21swapper/012:38:260
1502227618,39sleep015122-21apt-config09:09:430
1329627618,10sleep10-21swapper/110:44:471
107662766,12sleep30-21swapper/308:56:573
3212127522,26sleep20-21swapper/211:45:112
3162127513,30sleep30-21swapper/311:45:003
203327520,24sleep30-21swapper/308:29:433
1534027529,25sleep30-21swapper/307:25:153
148752758,50sleep10-21swapper/110:49:551
934327319,33sleep30-21swapper/308:51:563
638227313,44sleep06470-21dpkg10:24:430
638227313,44sleep06470-21dpkg10:24:430
632927318,31sleep36378-21dpkg10:24:423
632927318,31sleep36378-21dpkg10:24:423
3197427315,43sleep032043-21/usr/sbin/munin11:45:090
3115227314,18sleep00-21swapper/011:44:410
307332738,11sleep20-21swapper/209:56:442
2734927320,39sleep227433-21tail09:45:132
2048827320,32sleep20-21swapper/207:44:442
1546527325,30sleep30-21swapper/312:39:433
1341627324,29sleep30-21swapper/310:45:013
123792736,13sleep30-21swapper/309:00:053
123792736,13sleep30-21swapper/309:00:053
872327217,10sleep10-21swapper/112:17:081
51612729,52sleep25254-21latency_hist08:39:442
280127212,25sleep30-21swapper/311:55:143
2739627213,44sleep027432-21ps08:05:100
1154327222,28sleep20-21swapper/210:39:432
2676727118,21sleep00-21swapper/011:29:550
2392827117,35sleep024021-21apt-config11:19:420
118692717,45sleep211946-21sed08:59:432
585727016,38sleep35895-21grep08:40:093
2885127018,31sleep328884-21/usr/sbin/munin08:10:053
2885127018,31sleep328884-21/usr/sbin/munin08:10:053
2586227013,12sleep20-21swapper/208:00:112
710826912,39sleep37163-21/usr/sbin/munin08:45:023
308212699,50sleep00-21swapper/009:57:510
2530426922,27sleep30-21swapper/311:26:273
224526913,41sleep02263-21/usr/sbin/munin08:29:530
221982698,10sleep20-21swapper/211:12:292
1804826916,38sleep218056-21open_files07:35:152
1079426920,23sleep20-21swapper/207:14:442
1079426920,23sleep20-21swapper/207:14:442
2514326815,30sleep30-21swapper/307:59:433
1293926817,23sleep20-21swapper/207:19:572
819126716,26sleep10-21swapper/107:05:071
549326717,24sleep30-21swapper/312:05:033
2800426717,32sleep228007-21systemd-tmpfile11:34:252
1262026720,26sleep212648-21munin-run07:19:412
950326619,28sleep09591-21tune2fs07:10:060
950326619,28sleep09591-21tune2fs07:10:060
908426618,29sleep09115-21munin-run07:09:420
68322657,45sleep20-21swapper/210:24:532
3050826515,34sleep00-21swapper/008:15:110
3050826515,34sleep00-21swapper/008:15:110
269926518,20sleep20-21swapper/211:55:132
2410526517,30sleep224273-21apt-config11:19:442
171972657,12sleep20-21swapper/210:55:132
1676426518,22sleep30-21swapper/309:14:423
1067026527,21sleep30-21swapper/307:14:413
1067026527,21sleep30-21swapper/307:14:413
3259926414,24sleep20-21swapper/210:04:432
2293226414,26sleep30-21swapper/311:15:043
180226412,35sleep31851-21unin-run11:54:413
1718126421,23sleep20-21swapper/209:15:062
1289426410,40sleep312963-21expr12:30:103
2685026320,23sleep30-21swapper/311:29:593
263572638,10sleep00-21swapper/011:28:060
703126218,27sleep20-21swapper/208:45:012
3039826215,26sleep30-21swapper/308:15:063
3039826215,26sleep30-21swapper/308:15:063
1270126219,24sleep00-21swapper/007:19:430
88772614,45sleep30-21swapper/312:19:083
2134626118,24sleep20-21swapper/207:45:132
1106326121,21sleep20-21swapper/212:25:042
1020626113,35sleep010254-21awk10:34:570
100126117,22sleep20-21swapper/211:50:082
826926024,19sleep30-21swapper/307:05:093
1254926017,28sleep212317-21/usr/sbin/munin12:30:012
1175426017,19sleep20-21swapper/210:39:512
971025918,19sleep00-21swapper/012:20:060
834725923,21sleep20-21swapper/207:05:122
782125932,13sleep00-21swapper/010:25:520
652725916,27sleep26551-21munin-run12:09:422
2831425915,27sleep228310-21apt-get11:34:542
257325916,28sleep02627-21/usr/sbin/munin11:55:100
253825918,21sleep30-21swapper/308:30:043
2100025920,23sleep20-21swapper/209:25:152
1785825915,22sleep00-21swapper/007:35:090
1766025919,25sleep317669-21diskstats07:35:043
589925815,28sleep06015-21sed12:05:120
2693325814,27sleep226927-21apt-get08:04:552
2189825810,36sleep030095-21snmpd09:29:450
2039125815,28sleep219533-21/usr/sbin/munin11:05:142
160002588,13sleep20-21swapper/207:29:462
153412586,12sleep30-21swapper/310:50:103
48192576,10sleep30-21swapper/308:36:223
3162325715,26sleep231619-21apt-get08:19:572
2038225716,27sleep320417-21grep09:24:563
1125325713,28sleep311260-21/usr/sbin/munin12:25:093
698125612,26sleep26500-21apt-get12:09:502
416325614,24sleep34239-21mailstats12:00:103
374425617,18sleep20-21swapper/210:15:042
2637025618,24sleep024749-21kworker/u8:009:44:430
840725514,24sleep38400-21apt-get10:29:553
444125514,26sleep24566-21uptime12:00:132
328725514,25sleep23440-21sed10:14:442
3000025512,28sleep330097-21perl11:40:003
2945525514,24sleep229466-21munin-run09:54:422
2348225519,18sleep20-21swapper/209:34:562
2225225516,23sleep322248-21apt-get07:49:543
1642525515,24sleep016419-21apt-get10:54:550
1456225518,22sleep014459-21/usr/sbin/munin07:25:000
1409825518,16sleep00-21swapper/010:45:140
3062548,35sleep30-21swapper/308:23:213
2059425417,17sleep00-21swapper/009:25:050
607125314,24sleep26197-21cat08:40:122
266052539,20sleep30-21swapper/309:44:543
2395025315,21sleep20-21swapper/207:55:022
2387925318,20sleep00-21swapper/009:35:060
2255225318,21sleep20-21swapper/209:30:162
1997625313,24sleep320013-21ntp_states11:05:073
17452535,37sleep00-21swapper/011:53:570
1525725320,17sleep00-21swapper/010:50:070
924225212,23sleep29236-21apt-get12:19:512
3003125212,24sleep230025-21apt-get08:14:572
3003125212,24sleep230025-21apt-get08:14:572
2599225217,16sleep00-21swapper/008:00:130
2503925214,22sleep00-21swapper/011:25:030
1800025211,24sleep317997-21apt-get10:59:553
1440225217,16sleep30-21swapper/309:05:133
2892325111,25sleep329016-21sed11:35:113
2870225113,18sleep30-21swapper/309:50:113
229842516,35sleep30-21swapper/309:32:033
1247425116,17sleep00-21swapper/009:00:060
1247425116,17sleep00-21swapper/009:00:060
833925015,18sleep00-21swapper/008:49:430
367025012,23sleep03665-21apt-get11:59:550
2631625013,23sleep226323-21modprobe09:44:412
133725011,18sleep20-21swapper/208:25:142
1335425011,22sleep013391-21ntp_states07:20:070
1333025014,22sleep213322-21apt-get10:44:562
2513024912,15sleep30-21swapper/309:39:583
2070524914,17sleep30-21swapper/307:44:583
2724224815,16sleep30-21swapper/308:05:043
2147424814,22sleep021482-21needreboot11:10:050
133462484,31sleep20-21swapper/209:04:242
133462484,31sleep20-21swapper/209:04:242
1855124613,18sleep00-21swapper/011:00:120
1549224613,15sleep20-21swapper/210:50:132
1871624512,17sleep20-21swapper/211:00:142
900224414,16sleep20-21swapper/212:19:432
2578924410,17sleep20-21swapper/211:26:342
89222415,13sleep00-21swapper/012:19:410
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional