You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-05 - 03:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Sat Sep 05, 2026 00:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
202012234162,59sleep00-21swapper/019:06:170
202992233162,58sleep20-21swapper/219:07:322
202022226153,23sleep10-21swapper/119:06:181
185292224194,20sleep30-21swapper/319:05:083
549121820,4sleep02056899cyclictest22:14:550
524521360,4sleep12057299cyclictest22:11:471
56722550,4sleep339-21rcuc/321:00:143
200402530,2sleep20-21swapper/220:20:122
200402530,2sleep20-21swapper/220:20:122
23842520,4sleep230-21rcuc/223:20:152
103652520,1sleep10-21swapper/122:25:121
2058899404,6cyclictest0-21swapper/300:06:183
2058899384,6cyclictest25649-40dnf00:11:383
2058899381,11cyclictest27719-21/usr/sbin/munin23:05:253
2058899365,7cyclictest10573-40dnf21:10:243
2058899364,3cyclictest7581-21find23:30:213
2058899361,5cyclictest0-21swapper/320:15:203
2058899361,11cyclictest3533-21/usr/sbin/munin22:10:183
2058899353,9cyclictest827-21runrttasks22:38:023
2058899353,9cyclictest827-21runrttasks22:38:023
20588993520,4cyclictest0-21swapper/321:55:133
20588993511,3cyclictest28672-21/usr/sbin/munin20:40:223
2057299351,25cyclictest0-21swapper/122:16:291
2058899341,7cyclictest0-21swapper/319:45:313
20588993410,3cyclictest22084-21/usr/sbin/munin20:25:253
20581993431,2cyclictest0-21swapper/220:03:172
2057299341,3cyclictest0-21swapper/121:55:131
2057299341,15cyclictest11998-21mailstats23:40:231
2058899331,5cyclictest1974-21smart_sda20:50:243
2058899331,3cyclictest25022-21seq22:56:453
20588993310,3cyclictest827-21runrttasks23:39:543
2058199331,23cyclictest0-21swapper/221:00:442
2058199330,19cyclictest0-21swapper/220:05:172
2058899329,4cyclictest11045-21/usr/sbin/munin22:25:193
2058899328,6cyclictest21672-21proc_pri21:35:223
2058899327,4cyclictest0-21swapper/323:00:143
2058899322,10cyclictest26686-21cut19:20:183
20588993219,3cyclictest0-21swapper/319:25:573
20588993214,3cyclictest0-21swapper/320:23:053
20588993214,3cyclictest0-21swapper/320:23:053
20588993212,4cyclictest13784-21/usr/sbin/munin23:45:173
20588993211,3cyclictest827-21runrttasks21:17:453
20588993210,3cyclictest827-21runrttasks00:33:323
20588993210,3cyclictest26476-21/usr/sbin/munin20:35:203
2058899320,9cyclictest0-21swapper/322:20:243
2058899320,9cyclictest0-21swapper/320:05:253
2058899320,6cyclictest0-21swapper/323:27:593
2058899320,11cyclictest2900-21missed_timers23:20:193
2058199321,24cyclictest0-21swapper/220:27:512
20588993116,3cyclictest0-21swapper/300:37:573
2058899311,3cyclictest87350irq/126-enp1s0-20:00:013
20588993111,3cyclictest23831-21cut19:15:143
20588993111,3cyclictest18300-21/usr/sbin/munin21:30:133
20588993110,3cyclictest29062-21latency_hist21:55:013
20588993110,2cyclictest24894-21/usr/sbin/munin21:45:153
2058199311,12cyclictest0-21swapper/220:39:512
2057299311,2cyclictest0-21swapper/100:39:091
2057299311,13cyclictest0-21swapper/120:37:501
2056899317,11cyclictest28131-21munin-run00:20:010
2058899309,3cyclictest443-21if_err_enp2s023:15:163
2058899303,11cyclictest12233-21tr20:00:213
2058899301,6cyclictest0-21swapper/319:40:243
2058899301,5cyclictest8564-1kworker/u9:123:58:573
2058899301,5cyclictest0-21swapper/321:23:413
2058899301,4cyclictest31492-21/usr/sbin/munin22:00:193
20588993014,3cyclictest0-21swapper/319:33:293
2058899301,13cyclictest24967-21kworker/u8:123:10:253
2058899301,12cyclictest19534-21/usr/sbin/munin00:00:163
2058199301,12cyclictest0-21swapper/222:29:552
2058199300,23cyclictest0-21swapper/222:50:472
2058199300,19cyclictest0-21swapper/222:44:072
2057299302,15cyclictest10573-40dnf21:10:221
2057299301,7cyclictest0-21swapper/120:00:411
2057299301,3cyclictest0-21swapper/119:15:451
2057299301,18cyclictest0-21swapper/119:36:091
2057299301,12cyclictest0-21swapper/122:01:491
2057299300,20cyclictest0-21swapper/120:40:221
2058899299,3cyclictest22810-21tr21:40:113
2058899297,4cyclictest0-21swapper/322:42:253
2058899294,4cyclictest30733-21cstates00:25:133
2058899294,3cyclictest19966-21awk22:45:203
2058899294,3cyclictest0-21swapper/322:54:103
20588992911,3cyclictest2908-21/usr/sbin/munin20:55:143
2058899291,11cyclictest9278-21diskmemload00:16:213
20588992910,3cyclictest7898-21date22:20:013
20588992910,3cyclictest31105-21expr20:45:133
20588992910,2cyclictest827-21runrttasks21:28:553
20588992910,2cyclictest827-21runrttasks21:28:553
2058199291,15cyclictest0-21swapper/219:39:112
2058199291,10cyclictest0-21swapper/221:20:072
2058199290,12cyclictest0-21swapper/222:14:472
2057299291,7cyclictest0-21swapper/121:45:571
2057299291,12cyclictest0-21swapper/123:23:531
2057299291,12cyclictest0-21swapper/123:12:431
2058899281,3cyclictest87350irq/126-enp1s0-22:05:263
20588992811,3cyclictest0-21swapper/321:05:533
2058899281,11cyclictest9278-21diskmemload23:51:223
2058899280,5cyclictest0-21swapper/322:31:503
2058199281,13cyclictest0-21swapper/219:58:072
2058199281,13cyclictest0-21swapper/200:37:312
2058199281,12cyclictest0-21swapper/221:39:072
2058199281,12cyclictest0-21swapper/219:32:072
2058199281,10cyclictest0-21swapper/219:40:032
2058199280,13cyclictest0-21swapper/223:17:352
2058199280,13cyclictest0-21swapper/222:46:472
2058199280,12cyclictest0-21swapper/223:55:512
2058199280,12cyclictest0-21swapper/223:33:032
2058199280,12cyclictest0-21swapper/221:27:232
2058199280,12cyclictest0-21swapper/221:27:232
2057299281,18cyclictest0-21swapper/123:48:531
2057299281,14cyclictest0-21swapper/122:06:571
2057299281,13cyclictest0-21swapper/123:52:011
2057299281,13cyclictest0-21swapper/121:06:011
2057299281,12cyclictest0-21swapper/123:04:291
2057299281,12cyclictest0-21swapper/100:27:291
2057299281,11cyclictest0-21swapper/120:10:131
2057299281,11cyclictest0-21swapper/119:25:491
2057299281,11cyclictest0-21swapper/100:17:211
2056899288,17cyclictest20897-21munin-run22:50:010
20588992717,3cyclictest0-21swapper/323:43:453
2058199271,12cyclictest0-21swapper/221:46:432
2058199271,12cyclictest0-21swapper/220:19:192
2058199271,10cyclictest0-21swapper/221:56:152
2058199271,10cyclictest0-21swapper/219:46:312
2058199271,10cyclictest0-21swapper/219:14:312
2058199271,10cyclictest0-21swapper/200:08:312
2058199270,14cyclictest0-21swapper/223:12:512
2058199270,13cyclictest0-21swapper/223:04:072
2058199270,13cyclictest0-21swapper/200:02:072
2058199270,12cyclictest0-21swapper/223:27:312
2058199270,12cyclictest0-21swapper/221:52:472
2058199270,10cyclictest0-21swapper/200:31:152
2057299271,11cyclictest0-21swapper/121:53:531
2057299271,11cyclictest0-21swapper/120:06:491
2057299271,10cyclictest0-21swapper/123:58:251
2057299271,10cyclictest0-21swapper/123:39:131
2056899277,17cyclictest13688-21munin-run21:20:010
2056899274,11cyclictest3940-21idleruntime-cro23:25:010
2058899261,4cyclictest6685-21/usr/sbin/munin19:50:173
2058899260,4cyclictest0-21swapper/300:22:393
2058199261,9cyclictest0-21swapper/223:38:042
2058199261,9cyclictest0-21swapper/222:30:072
2058199261,12cyclictest0-21swapper/221:33:592
2058199261,12cyclictest0-21swapper/220:51:312
2058199261,11cyclictest0-21swapper/223:52:472
2058199261,11cyclictest0-21swapper/220:45:272
2058199261,10cyclictest0-21swapper/220:59:402
2058199261,10cyclictest0-21swapper/219:19:112
2058199260,13cyclictest0-21swapper/223:07:072
2058199260,12cyclictest0-21swapper/222:02:472
2057299269,3cyclictest0-21swapper/122:44:051
2057299261,6cyclictest0-21swapper/120:15:491
2057299261,16cyclictest13244-21memory22:30:211
2057299261,12cyclictest0-21swapper/120:58:371
2057299261,12cyclictest0-21swapper/120:20:291
2057299261,12cyclictest0-21swapper/120:20:291
2057299261,12cyclictest0-21swapper/119:54:011
2057299261,12cyclictest0-21swapper/119:23:571
2057299261,10cyclictest0-21swapper/121:43:451
2056899260,12cyclictest29083-21perf21:55:010
274512250,6sleep00-21swapper/020:35:200
2058899257,4cyclictest0-21swapper/319:35:253
2058899255,3cyclictest0-21swapper/319:11:333
2058199251,13cyclictest0-21swapper/221:11:332
2058199251,10cyclictest0-21swapper/220:34:112
2058199250,18cyclictest0-21swapper/222:38:032
2058199250,18cyclictest0-21swapper/222:38:032
2058199250,17cyclictest0-21swapper/223:48:442
2058199250,12cyclictest0-21swapper/221:40:312
2058199250,11cyclictest0-21swapper/222:57:192
2058199250,10cyclictest0-21swapper/222:05:032
2058199250,10cyclictest0-21swapper/219:24:592
2057299251,16cyclictest0-21swapper/121:21:021
2057299251,11cyclictest0-21swapper/122:22:331
2057299251,11cyclictest0-21swapper/121:28:051
2057299251,11cyclictest0-21swapper/121:28:051
2057299251,11cyclictest0-21swapper/119:48:131
2057299251,10cyclictest0-21swapper/123:31:411
2057299251,10cyclictest0-21swapper/122:46:541
2056899250,12cyclictest6113-21idleruntime-cro23:30:010
2058899246,3cyclictest0-21swapper/320:33:173
2058199241,9cyclictest0-21swapper/220:12:272
2058199241,11cyclictest0-21swapper/221:09:342
2058199240,11cyclictest0-21swapper/200:21:392
2058199240,10cyclictest0-21swapper/222:16:472
2058199240,10cyclictest0-21swapper/221:16:562
2057299248,3cyclictest0-21swapper/121:38:411
2057299248,3cyclictest0-21swapper/120:27:251
2057299247,7cyclictest0-21swapper/119:40:471
2057299247,4cyclictest0-21swapper/100:34:171
2057299241,9cyclictest0-21swapper/100:23:451
2057299241,12cyclictest5303-21expr21:00:111
2057299241,11cyclictest2181-21grep20:50:251
2057299241,11cyclictest0-21swapper/123:17:571
2057299241,10cyclictest0-21swapper/122:37:291
2057299241,10cyclictest0-21swapper/122:37:291
2057299241,10cyclictest0-21swapper/119:10:531
20572992410,3cyclictest0-21swapper/123:25:171
2056899241,11cyclictest24073-21idleruntime-cro20:30:010
2058899232,2cyclictest16482-21strings20:10:213
2058199238,3cyclictest0-21swapper/222:22:312
2058199230,10cyclictest0-21swapper/223:40:442
2057299239,2cyclictest22956-21taskset22:53:251
2057299238,3cyclictest0-21swapper/120:50:011
2057299238,3cyclictest0-21swapper/100:14:091
2057299237,3cyclictest0-21swapper/120:30:131
2057299233,3cyclictest0-21swapper/119:55:231
2057299231,16cyclictest0-21swapper/123:06:521
2057299231,14cyclictest0-21swapper/121:30:331
2057299231,10cyclictest0-21swapper/119:33:211
2057299231,10cyclictest0-21swapper/100:08:491
2056899238,9cyclictest0-21swapper/023:37:560
2056899231,6cyclictest0-21swapper/023:51:530
2056899231,11cyclictest31281-21perf22:00:010
2058199220,9cyclictest0-21swapper/200:19:392
2058199220,8cyclictest0-21swapper/219:53:592
2058199220,10cyclictest0-21swapper/200:14:472
2057299221,9cyclictest0-21swapper/121:15:111
2056899224,15cyclictest391-21basename20:50:010
2056899222,17cyclictest0-21swapper/021:10:220
2056899221,10cyclictest22286-21grep19:10:230
2056899221,10cyclictest0-21swapper/000:05:010
2056899215,14cyclictest23746-21basename00:10:010
2056899214,6cyclictest0-21swapper/021:25:160
2056899214,6cyclictest0-21swapper/021:25:160
20568992116,3cyclictest8254-21grep19:50:250
2056899211,5cyclictest0-21swapper/023:55:290
2056899211,11cyclictest26855-21grep19:20:200
2056899211,11cyclictest15878-21fschecks_count20:10:170
2056899211,10cyclictest25284-21latency_hist23:00:010
2058199204,3cyclictest0-21swapper/219:25:292
2058199201,10cyclictest0-21swapper/220:40:352
2058199200,10cyclictest0-21swapper/200:26:032
2057299206,3cyclictest0-21swapper/122:58:251
2057299201,11cyclictest0-21swapper/100:01:011
2056899205,13cyclictest11-21rcuc/000:35:250
2056899203,14cyclictest7-21ksoftirqd/021:05:190
2056899201,7cyclictest14666-21ethtool21:20:160
2056899201,6cyclictest0-21swapper/023:43:560
2056899201,11cyclictest5103-21/usr/sbin/munin21:00:150
2056899201,10cyclictest25539-21cpu23:00:110
16822201,2sleep00-21swapper/023:19:340
2056899199,3cyclictest3853-21sed19:40:250
2056899194,12cyclictest11-21rcuc/021:30:140
2056899192,9cyclictest13321-21grep22:30:220
2056899191,9cyclictest31384-21sshd23:10:250
2056899191,9cyclictest0-21swapper/019:55:120
2056899191,4cyclictest0-21swapper/023:33:390
2056899191,4cyclictest0-21swapper/022:52:400
2056899191,12cyclictest0-21swapper/019:30:120
268032181,2sleep00-21swapper/021:49:240
2056899186,3cyclictest0-21swapper/019:26:540
2056899184,5cyclictest0-21swapper/020:15:260
2056899183,12cyclictest11-21rcuc/022:00:220
2056899183,12cyclictest11-21rcuc/020:40:240
2056899183,12cyclictest11-21rcuc/000:20:160
2056899182,7cyclictest11082-21/usr/sbin/munin20:00:180
2056899182,6cyclictest989-21grep00:30:140
2056899181,9cyclictest0-21swapper/000:11:360
2056899181,3cyclictest0-21swapper/022:25:580
2056899181,10cyclictest0-21swapper/022:35:190
2056899181,10cyclictest0-21swapper/022:35:190
61862170,2sleep00-21swapper/019:46:400
2056899175,3cyclictest22089-21users21:35:260
2056899173,12cyclictest7-21ksoftirqd/019:35:110
2056899173,12cyclictest11-21rcuc/020:30:100
2056899171,7cyclictest24662-21needreboot19:15:210
2056899171,6cyclictest0-21swapper/023:49:400
2056899171,5cyclictest0-21swapper/023:07:290
20568991713,3cyclictest0-21swapper/020:53:240
20568991712,3cyclictest2908-21/usr/sbin/munin20:55:230
2056899171,13cyclictest81ktimersoftd/020:05:180
2056899171,13cyclictest6877-21head22:15:220
20568991711,3cyclictest17085-21fschecks_count22:40:130
2056899166,3cyclictest87550irq/128-enp1s0-00:25:150
2056899166,2cyclictest11-21rcuc/021:40:100
2056899165,4cyclictest1318-21/usr/sbin/munin22:05:150
2056899163,11cyclictest8045-21cut22:20:140
20568991510,3cyclictest19887-21acpi20:20:110
20568991510,3cyclictest19887-21acpi20:20:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional