You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-22 - 14:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot3s.osadl.org (updated Tue Sep 22, 2026 12:44:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
115252235162,59sleep20-21swapper/207:05:092
133112231163,55sleep30-21swapper/307:07:483
115192202166,24sleep10-21swapper/107:05:041
131642200162,26sleep00-21swapper/007:05:510
421421360,4sleep11357599cyclictest09:11:471
135829910097,2cyclictest0-21swapper/210:36:012
13582999488,2cyclictest0-21swapper/211:10:272
13582998077,2cyclictest0-21swapper/208:44:082
103352500,1sleep30-21swapper/310:40:133
241922490,1sleep10-21swapper/108:45:131
39292470,1sleep00-21swapper/010:25:180
23572460,2sleep10-21swapper/111:35:171
279192450,1sleep09-21rcu_preempt07:40:110
13569994013,13cyclictest10455-21munin-run08:15:010
253332370,1sleep20-21swapper/211:15:122
13569993610,12cyclictest14867-21basename08:25:010
1358299359,5cyclictest8038-21grep09:20:232
1358299341,6cyclictest0-21swapper/212:20:172
1356999349,12cyclictest16108-21basename10:55:010
142462330,4sleep339-21rcuc/308:20:243
1358299334,4cyclictest20121-21/usr/sbin/munin12:15:192
1358299331,25cyclictest0-21swapper/210:49:402
1358299331,25cyclictest0-21swapper/210:15:122
1358299331,19cyclictest0-21swapper/209:59:022
1358299321,6cyclictest5542-21cstates10:30:172
13582993213,3cyclictest28933-21grep08:55:172
1358299321,22cyclictest0-21swapper/212:04:282
1358299320,25cyclictest0-21swapper/211:40:002
269862310,4sleep011-21rcuc/012:30:230
1358699311,23cyclictest0-21swapper/309:13:023
1358699311,13cyclictest0-21swapper/307:27:503
1358699310,23cyclictest0-21swapper/312:35:103
1358299316,3cyclictest8564-1kworker/u9:107:42:152
1358299312,4cyclictest8257-21perf08:10:012
13582993115,3cyclictest0-21swapper/211:56:042
1358299311,24cyclictest0-21swapper/208:29:022
1358299311,24cyclictest0-21swapper/208:21:342
1358299311,15cyclictest0-21swapper/207:51:502
1358299311,15cyclictest0-21swapper/207:51:502
1358299311,14cyclictest0-21swapper/212:28:362
166702300,1sleep30-21swapper/307:15:133
1358299301,20cyclictest0-21swapper/208:34:022
1358299301,14cyclictest0-21swapper/211:45:362
1358299301,14cyclictest0-21swapper/209:16:422
1358299301,14cyclictest0-21swapper/207:48:352
1358299301,10cyclictest0-21swapper/210:55:042
1358299300,13cyclictest0-21swapper/208:54:142
1356999300,23cyclictest27950-21kworker/0:009:02:560
1358699291,17cyclictest0-21swapper/311:40:523
1358699290,22cyclictest0-21swapper/309:15:543
1358299291,14cyclictest0-21swapper/209:48:272
1358299291,14cyclictest0-21swapper/208:16:062
1358299290,13cyclictest0-21swapper/211:23:452
1357599291,6cyclictest0-21swapper/111:26:141
1358699281,21cyclictest0-21swapper/311:11:043
1358699281,14cyclictest0-21swapper/311:19:443
1358699281,12cyclictest0-21swapper/309:31:203
1358699281,11cyclictest0-21swapper/308:00:023
1358699281,11cyclictest0-21swapper/307:24:383
1358699280,22cyclictest0-21swapper/312:17:403
1358699280,17cyclictest0-21swapper/310:03:233
1358699280,10cyclictest0-21swapper/308:06:393
1358299289,3cyclictest0-21swapper/211:07:282
1358299286,10cyclictest0-21swapper/210:27:362
1358299281,5cyclictest18881-21grep07:20:122
13582992812,3cyclictest0-21swapper/209:00:062
1358299281,13cyclictest0-21swapper/212:07:322
1358299281,13cyclictest0-21swapper/209:11:222
1358299281,12cyclictest0-21swapper/207:28:062
1358299281,10cyclictest0-21swapper/207:16:022
13582992810,9cyclictest0-21swapper/209:42:112
1358299280,19cyclictest0-21swapper/208:02:502
1358299280,13cyclictest0-21swapper/211:02:322
1358699271,17cyclictest0-21swapper/307:12:503
1358699271,12cyclictest13557-21cyclictest09:40:443
1358699271,11cyclictest0-21swapper/311:37:003
1358699271,11cyclictest0-21swapper/308:31:443
1358699271,11cyclictest0-21swapper/307:57:543
1358699271,10cyclictest0-21swapper/307:51:143
1358699271,10cyclictest0-21swapper/307:51:143
1358699270,22cyclictest0-21swapper/311:06:443
1358699270,11cyclictest0-21swapper/311:48:513
1358699270,10cyclictest0-21swapper/310:53:563
1358699270,10cyclictest0-21swapper/310:15:323
1358299271,12cyclictest0-21swapper/210:41:242
1358299270,19cyclictest0-21swapper/212:39:482
1358299270,19cyclictest0-21swapper/209:54:382
1358299270,10cyclictest0-21swapper/207:32:182
13575992712,13cyclictest19314-21kworker/u8:007:30:051
13575992712,13cyclictest15280-21kworker/u8:209:59:431
1356999273,11cyclictest14536-21awk12:05:010
1356999272,13cyclictest22700-21munin-run11:10:010
1356999272,12cyclictest28670-21munin-run10:10:010
1358699261,9cyclictest0-21swapper/311:01:303
1358699261,9cyclictest0-21swapper/308:12:393
1358699261,18cyclictest0-21swapper/309:37:543
1358699261,15cyclictest0-21swapper/310:22:523
1358699261,11cyclictest0-21swapper/311:30:403
1358699261,10cyclictest0-21swapper/308:25:413
1358699260,9cyclictest0-21swapper/308:18:493
1358699260,12cyclictest0-21swapper/309:25:163
1358699260,12cyclictest0-21swapper/309:20:503
1358699260,10cyclictest0-21swapper/309:52:283
1358299267,3cyclictest0-21swapper/209:38:192
1358299261,5cyclictest7794-21kworker/u8:212:10:202
1358299261,14cyclictest0-21swapper/211:53:562
13582992611,3cyclictest0-21swapper/210:08:002
1356999263,11cyclictest2298-21idleruntime-cro09:10:010
1356999263,11cyclictest22080-21perf09:55:000
1356999262,10cyclictest18762-21kworker/0:012:37:390
1356999262,10cyclictest1114-21kworker/0:111:41:020
1356999261,8cyclictest21316-21kworker/0:108:41:010
210792250,1sleep10-21swapper/109:50:231
1358699251,9cyclictest0-21swapper/312:23:223
1358699251,9cyclictest0-21swapper/307:32:363
1358699251,8cyclictest0-21swapper/308:37:363
1358699251,14cyclictest0-21swapper/307:44:463
1358699251,10cyclictest0-21swapper/311:29:563
1358699251,10cyclictest0-21swapper/310:12:253
1358699251,10cyclictest0-21swapper/310:12:253
1358699251,10cyclictest0-21swapper/308:56:263
1358699250,18cyclictest0-21swapper/311:24:323
1358699250,18cyclictest0-21swapper/310:36:403
1358699250,11cyclictest0-21swapper/312:29:583
1358699250,11cyclictest0-21swapper/310:59:303
1358699250,11cyclictest0-21swapper/310:29:043
1358699250,10cyclictest0-21swapper/310:47:483
1358299258,3cyclictest0-21swapper/209:08:062
1358299251,5cyclictest32057-21dnf11:30:132
1358299251,14cyclictest0-21swapper/211:40:042
13582992511,3cyclictest0-21swapper/211:25:572
13582992511,3cyclictest0-21swapper/207:55:102
1358299251,12cyclictest0-21swapper/209:26:352
13582992510,3cyclictest0-21swapper/210:50:442
13582992510,3cyclictest0-21swapper/210:21:202
1357599251,4cyclictest5102-21turbostat10:30:011
13575992514,5cyclictest28435-21kworker/u8:110:22:111
1357599251,16cyclictest0-21swapper/112:15:161
1356999259,7cyclictest2644-21kworker/0:010:40:100
1356999252,10cyclictest13080-21kworker/0:109:37:040
1356999251,8cyclictest2644-21kworker/0:010:33:350
1356999251,8cyclictest1285-21kworker/0:308:09:300
13569992518,4cyclictest17229-21grep09:40:240
312722241,3sleep131432-21users07:45:261
1358699244,3cyclictest0-21swapper/309:49:243
1358699241,10cyclictest0-21swapper/307:37:103
1358699240,9cyclictest0-21swapper/312:34:023
1358699240,9cyclictest0-21swapper/310:30:303
1358699240,9cyclictest0-21swapper/308:51:503
1358699240,12cyclictest0-21swapper/312:12:003
1358699240,11cyclictest0-21swapper/310:09:513
1358699240,10cyclictest0-21swapper/311:57:393
1358299248,4cyclictest0-21swapper/212:32:042
1358299248,3cyclictest0-21swapper/207:12:142
1358299246,7cyclictest0-21swapper/208:48:522
1358299241,5cyclictest0-21swapper/209:30:042
1357599240,17cyclictest0-21swapper/109:20:221
1356999245,16cyclictest7914-21munin-run11:50:010
1356999242,9cyclictest22852-21kworker/0:107:37:020
1356999242,9cyclictest20633-21kworker/0:207:29:200
1356999242,8cyclictest1285-21kworker/0:308:01:080
1356999241,9cyclictest30010-21kworker/0:109:15:120
1356999241,9cyclictest28317-21kworker/0:210:12:300
1356999241,9cyclictest28317-21kworker/0:210:12:300
1356999241,8cyclictest8581-21kworker/0:109:22:280
1356999241,10cyclictest19092-21kworker/0:008:35:120
1356999240,9cyclictest13878-21kworker/0:010:56:140
240472231,3sleep124165-21mailstats11:10:221
1358699231,9cyclictest0-21swapper/312:03:333
1358699231,9cyclictest0-21swapper/307:48:083
1358699231,10cyclictest0-21swapper/311:51:053
1358699231,10cyclictest0-21swapper/308:48:223
1358699230,10cyclictest0-21swapper/312:09:433
1358299235,9cyclictest9640-21proc_pri08:10:242
13582992310,3cyclictest0-21swapper/208:38:302
1357599231,17cyclictest0-21swapper/110:13:291
1357599231,17cyclictest0-21swapper/110:13:291
1357599231,11cyclictest9213-21ntp_states11:50:241
1356999231,6cyclictest0-21swapper/011:20:130
1356999231,10cyclictest24987-21sed12:25:260
1356999230,11cyclictest12323-21perf12:00:010
1358699220,9cyclictest0-21swapper/309:56:453
1358299229,3cyclictest0-21swapper/207:35:392
1358299221,14cyclictest0-21swapper/210:12:562
1358299221,14cyclictest0-21swapper/210:12:562
1357599229,11cyclictest28435-21kworker/u8:110:30:351
1357599222,18cyclictest0-21swapper/108:26:431
1357599222,15cyclictest0-21swapper/111:45:251
1357599221,3cyclictest28943-21hddtemp_smartct08:55:181
1357599221,16cyclictest0-21swapper/111:58:571
1357599221,16cyclictest0-21swapper/111:21:431
1357599221,16cyclictest0-21swapper/110:49:261
1356999229,3cyclictest18762-21kworker/0:012:17:060
1356999227,6cyclictest0-21swapper/010:01:080
1356999223,2cyclictest20633-21kworker/0:207:23:440
1356999222,8cyclictest2644-21kworker/0:010:22:460
1356999222,8cyclictest25106-21kworker/0:011:28:580
1356999222,8cyclictest1285-21kworker/0:308:15:320
1356999222,11cyclictest1285-21kworker/0:308:27:080
1356999221,9cyclictest24146-21kworker/0:009:59:020
1356999221,9cyclictest1285-21kworker/0:308:33:200
1356999221,11cyclictest23582-21kworker/0:208:48:150
13569992210,2cyclictest1285-21kworker/0:307:57:560
1358699219,3cyclictest0-21swapper/308:44:303
1358699210,9cyclictest0-21swapper/309:08:583
1357599219,5cyclictest6660-21kworker/u8:109:37:431
1357599214,9cyclictest0-21swapper/112:33:471
1357599212,15cyclictest10169-21ntp_states09:25:211
1357599212,14cyclictest0-21swapper/112:01:371
1357599211,9cyclictest0-21swapper/112:12:071
13575992117,3cyclictest0-21swapper/109:05:241
1357599211,18cyclictest0-21swapper/107:22:251
1357599211,15cyclictest0-21swapper/112:09:541
1357599211,14cyclictest0-21swapper/112:21:321
1357599211,14cyclictest0-21swapper/111:16:381
1357599211,14cyclictest0-21swapper/109:49:511
1357599211,11cyclictest8584-21ntp_states10:35:201
1357599211,10cyclictest19314-21kworker/u8:007:35:501
1357599211,10cyclictest0-21swapper/110:04:251
1356999216,6cyclictest0-21swapper/009:29:170
1356999212,11cyclictest18762-21kworker/0:012:14:020
1356999211,9cyclictest28317-21kworker/0:210:15:140
1356999211,8cyclictest3536-21kworker/0:009:33:040
1356999211,8cyclictest3536-21kworker/0:009:14:000
1356999211,11cyclictest27950-21kworker/0:008:54:290
1356999211,11cyclictest18037-21kworker/0:111:04:460
1356999210,9cyclictest3247-21kworker/0:311:38:020
1356999210,9cyclictest12885-21kworker/0:207:10:020
1356999210,11cyclictest15410-21kworker/0:107:17:010
1358699201,12cyclictest0-21swapper/309:01:463
1358299204,3cyclictest0-21swapper/210:02:062
1357599202,16cyclictest0-21swapper/112:25:371
1357599202,16cyclictest0-21swapper/111:44:151
1357599201,3cyclictest0-21swapper/108:18:531
13575992012,3cyclictest0-21swapper/112:35:061
1357599201,17cyclictest0-21swapper/108:05:181
13575992011,5cyclictest0-21swapper/109:42:061
1357599201,14cyclictest0-21swapper/111:07:431
1357599201,14cyclictest0-21swapper/110:58:161
1357599201,11cyclictest6677-21munin-run09:20:021
1357599201,11cyclictest32227-21fschecks_time11:30:141
1357599201,11cyclictest22586-21memory08:40:191
1356999207,3cyclictest22852-21kworker/0:107:30:580
1356999204,2cyclictest31262-21kworker/0:211:31:180
1356999201,8cyclictest31670-21kworker/0:207:50:120
1356999201,8cyclictest31670-21kworker/0:207:50:120
1356999201,8cyclictest27950-21kworker/0:008:56:320
1356999201,11cyclictest17898-21/usr/sbin/munin09:45:270
1356999201,11cyclictest14750-21/usr/sbin/munin12:05:260
1356999201,10cyclictest22520-21kworker/u8:211:15:310
1357599192,3cyclictest0-21swapper/108:30:551
1357599192,3cyclictest0-21swapper/107:53:091
1357599192,3cyclictest0-21swapper/107:53:091
1357599192,16cyclictest0-21swapper/109:30:321
1357599192,16cyclictest0-21swapper/108:01:241
13575991912,2cyclictest0-21swapper/111:00:411
1357599191,17cyclictest0-21swapper/110:53:321
1357599191,17cyclictest0-21swapper/110:07:131
1357599191,17cyclictest0-21swapper/108:50:391
1357599191,17cyclictest0-21swapper/108:39:391
1357599191,17cyclictest0-21swapper/108:21:091
1357599191,17cyclictest0-21swapper/107:25:501
1357599191,10cyclictest14878-21pluginstate07:10:221
1356999197,2cyclictest22852-21kworker/0:107:47:460
1356999197,2cyclictest1114-21kworker/0:111:53:140
1356999191,11cyclictest0-21swapper/011:13:060
1356999191,10cyclictest9375-21kworker/0:110:46:020
95132180,2sleep10-21swapper/108:10:231
1357599187,3cyclictest16890-21expr07:15:141
1357599183,9cyclictest0-21swapper/110:44:091
1357599182,3cyclictest0-21swapper/109:00:241
13575991815,2cyclictest0-21swapper/107:57:041
1357599181,16cyclictest0-21swapper/107:40:261
1356999185,3cyclictest88950irq/132-enp2s0-10:38:190
1357599171,14cyclictest0-21swapper/110:15:561
1356999171,13cyclictest21338-21/usr/sbin/munin12:20:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional