You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-19 - 18:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Sun Jul 19, 2026 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
256696321210,3sleep03056-21PK-Backend10:55:230
258614421150,2sleep23056-21PK-Backend11:50:212
255082421080,3sleep13056-21PK-Backend10:10:161
248478928156,17sleep10-21swapper/107:05:151
248572628058,14sleep00-21swapper/007:07:260
248576027839,14sleep20-21swapper/207:07:532
248559726432,26sleep30-21swapper/307:05:443
24994602620,3sleep13056-21PK-Backend07:45:011
25948022530,4sleep33056-21PK-Backend12:15:213
25222532520,4sleep13056-21PK-Backend08:50:011
25222532520,4sleep13056-21PK-Backend08:50:001
25856532480,3sleep30-21swapper/311:50:143
25033102410,3sleep30-21swapper/307:55:133
25720782340,3sleep20-21swapper/211:10:212
139099310,21rtkit-daemon0-21swapper/012:10:100
248614399289,17cyclictest2495911-21sed07:35:011
25756252270,12sleep234-21ksoftirqd/211:20:232
25589272270,5sleep30-21swapper/310:35:003
248615499270,24cyclictest2552636-21/usr/sbin/munin10:15:163
2486154992620,5cyclictest42-21ksoftirqd/310:03:423
248614399267,5cyclictest2597739-21cat12:25:141
2486143992610,14cyclictest2538682-21ntp_states09:35:221
139099260,4rtkit-daemon1389-21rtkit-daemon12:23:080
25829142250,9sleep126-21ksoftirqd/111:40:301
25829142250,9sleep126-21ksoftirqd/111:40:301
248614399259,14cyclictest2492003-21unattended-upgr07:21:521
248614399256,4cyclictest2547276-21basename10:00:161
248614399254,5cyclictest2507231-21cut08:05:181
2486143992510,13cyclictest2532459-21sh09:20:001
248614399250,1cyclictest3056-21PK-Backend09:50:181
248614399250,1cyclictest0-21swapper/107:29:311
139099250,17rtkit-daemon3056-21PK-Backend11:28:121
248614399248,5cyclictest2590953-21irqstats12:05:161
248614399245,4cyclictest2593817-21latency_hist12:15:001
2486143992420,3cyclictest3056-21PK-Backend11:05:211
248614399241,8cyclictest2487601-21/usr/sbin/munin07:10:231
2486143992410,13cyclictest2555509-21/usr/sbin/munin10:25:241
139099240,16rtkit-daemon3056-21PK-Backend11:24:391
25362012230,10chrt34-21ksoftirqd/209:30:112
248615499230,9cyclictest25207731cpuspeed_turbos08:45:133
248615499230,9cyclictest25207731cpuspeed_turbos08:45:133
2486151992321,1cyclictest0-21swapper/212:19:242
248615199230,21cyclictest2518216-21users08:35:222
248614399239,12cyclictest25804552sleep111:35:141
248614399237,5cyclictest2496325-21expr07:35:131
248614399236,6cyclictest2587800-21ps11:55:201
248614399236,15cyclictest0-21swapper/109:40:131
248614399235,4cyclictest2589947-21seq12:01:501
248614399231,9cyclictest2514041-21df08:25:141
2486139992321,1cyclictest0-21swapper/008:25:580
2486139992313,7cyclictest883-21dbus-daemon08:54:590
139099230,15rtkit-daemon3056-21PK-Backend09:31:511
2486154992220,1cyclictest3056-21PK-Backend07:31:383
2486154992220,1cyclictest3056-21PK-Backend07:18:533
2486154992220,1cyclictest0-21swapper/309:48:463
2486154992220,1cyclictest0-21swapper/308:34:373
2486154992219,2cyclictest3056-21PK-Backend12:20:343
248615499220,21cyclictest0-21swapper/307:46:523
2486151992220,1cyclictest0-21swapper/210:51:232
2486151992220,1cyclictest0-21swapper/209:12:262
2486151992220,1cyclictest0-21swapper/208:56:592
248615199220,21cyclictest0-21swapper/208:49:492
248615199220,21cyclictest0-21swapper/208:49:482
248614399227,3cyclictest2490373-21latency_hist07:20:011
248614399227,2cyclictest2599340-21sh12:30:011
248614399225,4cyclictest2566759-21cut10:55:211
248614399225,4cyclictest2526135-21cat09:00:131
2486143992220,1cyclictest3056-21PK-Backend10:48:591
2486143992219,2cyclictest3056-21PK-Backend11:49:431
248614399220,9cyclictest2511674-21ls08:20:011
248614399220,7cyclictest2585981-21/usr/sbin/munin11:50:201
248614399220,21cyclictest3056-21PK-Backend07:48:581
248614399220,20cyclictest3056-21PK-Backend11:01:361
248614399220,1cyclictest3056-21PK-Backend07:51:341
2486139992220,1cyclictest0-21swapper/008:58:550
2486139992215,5cyclictest2502978-21cron07:55:000
248613999220,21cyclictest0-21swapper/007:57:450
248613999220,20cyclictest0-21swapper/008:15:090
139099220,16rtkit-daemon3056-21PK-Backend08:13:111
139099220,16rtkit-daemon0-21swapper/009:44:250
139099220,14rtkit-daemon0-21swapper/007:32:160
2486154992119,1cyclictest0-21swapper/311:21:193
2486154992119,1cyclictest0-21swapper/311:17:343
2486154992119,1cyclictest0-21swapper/311:01:463
2486154992119,1cyclictest0-21swapper/310:57:123
2486154992119,1cyclictest0-21swapper/310:48:543
2486154992119,1cyclictest0-21swapper/309:52:303
2486154992119,1cyclictest0-21swapper/309:29:053
2486154992119,1cyclictest0-21swapper/309:12:163
2486154992119,1cyclictest0-21swapper/308:14:083
2486154992118,2cyclictest2510200-21apt08:15:113
2486154992112,6cyclictest436-20systemd-journal09:25:013
248615499210,20cyclictest0-21swapper/312:34:483
248615499210,20cyclictest0-21swapper/310:52:403
248615499210,20cyclictest0-21swapper/309:56:153
248615199213,16cyclictest2490206-21cron07:20:002
2486151992119,1cyclictest0-21swapper/212:10:182
2486151992119,1cyclictest0-21swapper/212:08:422
2486151992119,1cyclictest0-21swapper/212:03:202
2486151992119,1cyclictest0-21swapper/211:58:002
2486151992119,1cyclictest0-21swapper/211:48:072
2486151992119,1cyclictest0-21swapper/211:33:402
2486151992119,1cyclictest0-21swapper/211:08:452
2486151992119,1cyclictest0-21swapper/211:00:272
2486151992119,1cyclictest0-21swapper/210:31:342
2486151992119,1cyclictest0-21swapper/210:27:372
2486151992119,1cyclictest0-21swapper/209:51:342
2486151992119,1cyclictest0-21swapper/209:47:492
2486151992119,1cyclictest0-21swapper/209:38:082
2486151992119,1cyclictest0-21swapper/209:23:352
2486151992119,1cyclictest0-21swapper/208:24:542
2486151992119,1cyclictest0-21swapper/208:10:482
2486151992119,1cyclictest0-21swapper/208:09:472
2486151992119,1cyclictest0-21swapper/207:58:382
2486151992114,5cyclictest2523277-21threads08:50:202
248615199210,20cyclictest0-21swapper/212:26:192
248615199210,20cyclictest0-21swapper/210:00:392
248615199210,20cyclictest0-21swapper/209:25:452
248615199210,20cyclictest0-21swapper/207:55:002
248615199210,20cyclictest0-21swapper/207:31:412
248614399219,11cyclictest3056-21PK-Backend09:13:301
248614399217,3cyclictest2573638-21open_files11:15:191
248614399216,2cyclictest0-21swapper/108:30:211
248614399214,3cyclictest2594465-21latency12:15:181
2486143992119,1cyclictest3056-21PK-Backend10:31:491
2486143992119,1cyclictest3056-21PK-Backend10:20:201
2486143992119,1cyclictest3056-21PK-Backend08:40:361
2486143992118,2cyclictest3056-21PK-Backend08:53:261
248614399211,6cyclictest2597151-21turbostat.cron12:25:011
248614399210,8cyclictest2546736-21sed10:00:011
248614399210,5cyclictest436-20systemd-journal08:25:001
248614399210,19cyclictest3056-21PK-Backend07:56:481
2486139992119,1cyclictest3056-21PK-Backend10:00:340
2486139992119,1cyclictest0-21swapper/012:36:080
2486139992119,1cyclictest0-21swapper/011:32:000
2486139992119,1cyclictest0-21swapper/010:34:550
2486139992119,1cyclictest0-21swapper/009:53:510
2486139992119,1cyclictest0-21swapper/009:01:190
2486139992119,1cyclictest0-21swapper/008:23:000
2486139992119,1cyclictest0-21swapper/007:45:420
248613999210,20cyclictest0-21swapper/008:45:520
248613999210,20cyclictest0-21swapper/008:45:520
248613999210,20cyclictest0-21swapper/008:04:000
248613999210,19cyclictest0-21swapper/011:10:090
139099210,15rtkit-daemon0-21swapper/110:15:261
139099210,14rtkit-daemon3056-21PK-Backend09:05:071
2486154992018,1cyclictest3056-21PK-Backend09:36:223
2486154992018,1cyclictest0-21swapper/311:43:513
2486154992018,1cyclictest0-21swapper/311:43:503
2486154992018,1cyclictest0-21swapper/311:10:523
2486154992018,1cyclictest0-21swapper/310:40:513
2486154992018,1cyclictest0-21swapper/310:36:183
2486154992018,1cyclictest0-21swapper/310:22:593
2486154992018,1cyclictest0-21swapper/310:10:433
2486154992018,1cyclictest0-21swapper/308:06:383
2486154992018,1cyclictest0-21swapper/307:43:263
2486154992018,1cyclictest0-21swapper/307:35:493
2486154992015,3cyclictest2569831-21ls11:05:163
2486154992013,5cyclictest2591136-21needreboot12:05:183
2486154992013,5cyclictest2503150-21latency_hist07:55:013
248615499200,1cyclictest3056-21PK-Backend12:01:493
248615499200,19cyclictest3056-21PK-Backend12:13:043
248615499200,19cyclictest3056-21PK-Backend07:11:363
248615499200,19cyclictest0-21swapper/312:36:233
248615499200,19cyclictest0-21swapper/311:28:013
248615499200,19cyclictest0-21swapper/310:09:073
2486151992018,1cyclictest0-21swapper/212:37:482
2486151992018,1cyclictest0-21swapper/212:20:112
2486151992018,1cyclictest0-21swapper/210:40:552
2486151992018,1cyclictest0-21swapper/210:35:332
2486151992018,1cyclictest0-21swapper/208:43:392
2486151992018,1cyclictest0-21swapper/208:28:402
2486151992015,4cyclictest2492267-21cron07:25:012
2486151992015,3cyclictest978-21snmpd09:40:072
248615199200,1cyclictest0-21swapper/212:31:392
248615199200,1cyclictest0-21swapper/210:10:332
248615199200,1cyclictest0-21swapper/207:28:172
248615199200,19cyclictest0-21swapper/211:42:472
248615199200,19cyclictest0-21swapper/211:42:472
248615199200,19cyclictest0-21swapper/210:16:422
248615199200,19cyclictest0-21swapper/210:05:112
248615199200,19cyclictest0-21swapper/209:59:042
248615199200,19cyclictest0-21swapper/208:32:252
248615199200,19cyclictest0-21swapper/207:47:562
248614399204,3cyclictest0-21swapper/108:05:011
2486143992018,1cyclictest3056-21PK-Backend09:24:561
248614399201,7cyclictest2524933-21sensors_temp08:55:221
2486143992017,2cyclictest3056-21PK-Backend09:45:101
248614399200,7cyclictest2572629-21perf11:15:001
248614399200,7cyclictest2561457-21awk10:40:181
248614399200,19cyclictest3056-21PK-Backend11:34:081
2486139992018,1cyclictest0-21swapper/012:02:560
2486139992018,1cyclictest0-21swapper/011:43:360
2486139992018,1cyclictest0-21swapper/011:43:350
2486139992018,1cyclictest0-21swapper/011:17:260
2486139992018,1cyclictest0-21swapper/010:06:290
2486139992018,1cyclictest0-21swapper/009:18:230
2486139992018,1cyclictest0-21swapper/009:14:170
2486139992018,1cyclictest0-21swapper/007:40:420
2486139992018,1cyclictest0-21swapper/007:11:150
2486139992017,2cyclictest3056-21PK-Backend09:34:590
2486139992015,3cyclictest2516880-21munin-plugin-st08:35:000
248613999200,19cyclictest0-21swapper/012:31:350
248613999200,19cyclictest0-21swapper/012:06:410
248613999200,19cyclictest0-21swapper/011:21:450
248613999200,19cyclictest0-21swapper/009:56:480
248613999200,19cyclictest0-21swapper/009:37:090
248613999200,19cyclictest0-21swapper/009:26:330
248613999200,19cyclictest0-21swapper/008:44:160
248613999200,19cyclictest0-21swapper/007:23:040
248613999200,19cyclictest0-21swapper/007:15:340
25871842190,4sleep33056-21PK-Backend11:55:133
24965052190,4sleep03056-21PK-Backend07:35:160
248615499192,12cyclictest2527540-21apt09:05:123
2486154991917,1cyclictest0-21swapper/311:47:353
2486154991917,1cyclictest0-21swapper/308:35:243
2486154991916,2cyclictest3056-21PK-Backend12:28:043
248615499191,4cyclictest2537695-21munin-run09:34:593
2486154991914,3cyclictest2555907-21df_inode10:25:133
2486154991910,7cyclictest1-21systemd09:20:013
248615499190,1cyclictest3056-21PK-Backend08:26:333
248615499190,1cyclictest3056-21PK-Backend08:20:043
248615499190,1cyclictest0-21swapper/311:31:483
248615499190,1cyclictest0-21swapper/308:59:113
248615499190,18cyclictest3056-21PK-Backend07:22:383
248615499190,18cyclictest0-21swapper/308:41:193
2486151991917,1cyclictest0-21swapper/211:15:432
2486151991917,1cyclictest0-21swapper/210:59:402
248615199190,1cyclictest3056-21PK-Backend07:40:192
248615199190,1cyclictest0-21swapper/210:24:262
248615199190,1cyclictest0-21swapper/209:04:082
248615199190,17cyclictest2563240-21ntpq10:45:212
2486143991916,2cyclictest3056-21PK-Backend08:35:291
248614399190,7cyclictest2601796-21hddtemp_smartct12:35:141
248614399190,7cyclictest2560578-21sed10:40:001
248614399190,7cyclictest2535184-21ntpq09:25:191
248614399190,1cyclictest3056-21PK-Backend10:05:531
248613999192,15cyclictest2554468-21grep10:20:170
2486139991917,1cyclictest0-21swapper/011:59:110
2486139991917,1cyclictest0-21swapper/011:26:530
2486139991917,1cyclictest0-21swapper/011:03:270
2486139991917,1cyclictest0-21swapper/010:18:180
248613999190,1cyclictest3056-21PK-Backend08:10:300
248613999190,1cyclictest0-21swapper/010:13:460
248613999190,18cyclictest0-21swapper/010:41:370
2486154991816,1cyclictest0-21swapper/308:50:393
248615499180,1cyclictest0-21swapper/308:01:583
2486151991816,1cyclictest0-21swapper/209:08:202
2486139991816,1cyclictest0-21swapper/011:07:130
2486139991816,1cyclictest0-21swapper/010:37:180
2486139991816,1cyclictest0-21swapper/009:45:120
2486139991811,5cyclictest2599129-21cut12:30:000
2486139991811,5cyclictest2028-21upowerd10:28:590
248613999180,1cyclictest0-21swapper/012:17:550
248613999180,1cyclictest0-21swapper/011:46:320
248615499178,8cyclictest25257801/usr/sbin/munin09:00:173
2486154991711,5cyclictest2580999-21cat11:35:213
248615199179,6cyclictest2531164-21expr09:15:132
2486151991715,1cyclictest0-21swapper/208:19:472
2486151991715,1cyclictest0-21swapper/207:14:112
2486151991712,4cyclictest2497011-21cat07:35:222
2486151991711,4cyclictest2580072-21cat11:35:092
248614399175,3cyclictest3056-21PK-Backend10:51:191
2486139991715,1cyclictest0-21swapper/011:50:180
2486139991715,1cyclictest0-21swapper/010:46:430
2486139991715,1cyclictest0-21swapper/008:36:580
2486139991715,1cyclictest0-21swapper/008:05:010
248615499165,10cyclictest42-21ksoftirqd/307:29:593
2486154991614,1cyclictest0-21swapper/309:43:393
2486151991612,3cyclictest2504992-21sh08:00:112
2486139991614,1cyclictest0-21swapper/009:23:490
248615199159,5cyclictest2576881-21sed11:25:172
2486139991513,1cyclictest0-21swapper/010:51:500
2486139991510,4cyclictest2493410-21python307:25:210
248613999149,3cyclictest2527997-21if_err_enp2s009:05:170
248613999146,6cyclictest436-20systemd-journal11:40:000
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional