You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-03 - 17:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Mon Aug 03, 2026 12:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
270873521050,4sleep00-21swapper/012:35:160
26141672970,3sleep30-21swapper/308:05:203
25921072840,16chrt131ksoftirqd/007:05:210
139099830,73rtkit-daemon0-21swapper/307:09:523
259279726444,14sleep20-21swapper/207:09:272
259257826343,14sleep10-21swapper/107:06:361
26869752620,2sleep30-21swapper/311:35:123
139099590,27rtkit-daemon1389-21rtkit-daemon07:26:392
26936992560,4sleep23056-21PK-Backend11:55:002
26854552490,1sleep30-21swapper/311:30:113
26651192470,2sleep20-21swapper/210:30:242
27035252450,3sleep30-21swapper/312:20:153
26804232450,4chrt3056-21PK-Backend11:15:160
26456882400,3sleep20-21swapper/209:35:172
26174702240,3sleep2331rcuc/208:15:182
25976302240,8sleep126-21ksoftirqd/107:20:041
2593043992314,7cyclictest2629187-21apt08:50:112
2593038992321,1cyclictest3056-21PK-Backend11:43:331
2593036992311,10cyclictest2673620-21memory10:55:180
259303699230,21cyclictest2603252-21diskstats07:35:140
139099230,16rtkit-daemon3056-21PK-Backend12:08:503
2593048992216,4cyclictest436-20systemd-journal09:39:553
259304399224,15cyclictest2682357-21needreboot11:20:172
2593043992220,1cyclictest0-21swapper/210:56:542
2593043992220,1cyclictest0-21swapper/210:28:282
2593043992214,6cyclictest2700792-21sshd12:11:592
259304399220,20cyclictest2684185-21ntp_states11:25:202
2593038992219,2cyclictest3056-21PK-Backend08:11:561
2593038992219,2cyclictest3056-21PK-Backend08:11:551
2593038992215,5cyclictest2644523-21sshd09:31:581
259303899220,20cyclictest3056-21PK-Backend12:30:311
259303899220,20cyclictest3056-21PK-Backend11:19:471
2593036992220,1cyclictest0-21swapper/009:39:310
139099220,14rtkit-daemon0-21swapper/312:10:143
259304899211,8cyclictest2669937-21df10:45:123
2593048992114,5cyclictest2627687-21expr08:45:143
2593048992113,6cyclictest2608796-21grep07:50:183
2593048992113,6cyclictest1-21systemd08:00:003
259304899211,18cyclictest2622733-21irqstats08:30:173
259304899211,17cyclictest2663239-21python310:25:203
259304899210,2cyclictest0-21swapper/310:20:133
259304899210,20cyclictest2613089-21unin-run08:05:013
259304899210,20cyclictest0-21swapper/311:08:523
259304899210,1cyclictest0-21swapper/309:53:353
259304899210,19cyclictest42-21ksoftirqd/309:55:143
2593043992119,1cyclictest0-21swapper/212:35:362
2593043992119,1cyclictest0-21swapper/212:33:332
2593043992119,1cyclictest0-21swapper/212:26:302
2593043992119,1cyclictest0-21swapper/212:21:302
2593043992119,1cyclictest0-21swapper/212:15:422
2593043992119,1cyclictest0-21swapper/212:00:282
2593043992119,1cyclictest0-21swapper/211:06:202
2593043992119,1cyclictest0-21swapper/211:04:462
2593043992119,1cyclictest0-21swapper/210:48:432
2593043992119,1cyclictest0-21swapper/209:30:362
2593043992119,1cyclictest0-21swapper/207:58:152
2593043992119,1cyclictest0-21swapper/207:11:522
259304399210,20cyclictest0-21swapper/211:44:122
259304399210,20cyclictest0-21swapper/210:51:262
259304399210,20cyclictest0-21swapper/210:44:372
259304399210,20cyclictest0-21swapper/210:21:522
259304399210,20cyclictest0-21swapper/209:50:362
259304399210,20cyclictest0-21swapper/209:40:022
259304399210,20cyclictest0-21swapper/208:43:442
259304399210,20cyclictest0-21swapper/208:27:422
259304399210,20cyclictest0-21swapper/208:00:252
259304399210,20cyclictest0-21swapper/207:16:582
2593038992119,1cyclictest3056-21PK-Backend12:26:321
2593038992119,1cyclictest3056-21PK-Backend12:15:571
2593038992119,1cyclictest3056-21PK-Backend10:32:281
2593038992119,1cyclictest3056-21PK-Backend10:21:261
2593038992119,1cyclictest3056-21PK-Backend08:48:461
2593038992119,1cyclictest3056-21PK-Backend08:05:261
2593038992119,1cyclictest0-21swapper/109:08:531
2593038992119,1cyclictest0-21swapper/108:44:551
2593038992118,2cyclictest3056-21PK-Backend11:48:331
2593038992118,2cyclictest3056-21PK-Backend10:00:041
2593038992118,2cyclictest3056-21PK-Backend09:48:551
2593038992111,9cyclictest2643073-21unin-run09:30:001
259303899210,20cyclictest3056-21PK-Backend07:52:081
259303899210,20cyclictest0-21swapper/110:40:271
259303899210,20cyclictest0-21swapper/107:27:411
259303899210,1cyclictest3056-21PK-Backend07:38:021
259303899210,19cyclictest3056-21PK-Backend12:37:131
259303899210,19cyclictest3056-21PK-Backend09:58:081
2593036992119,1cyclictest0-21swapper/011:52:010
2593036992119,1cyclictest0-21swapper/011:41:120
2593036992119,1cyclictest0-21swapper/011:37:000
2593036992119,1cyclictest0-21swapper/010:15:080
2593036992119,1cyclictest0-21swapper/009:14:300
2593036992119,1cyclictest0-21swapper/008:54:570
2593036992119,1cyclictest0-21swapper/008:02:180
2593036992119,1cyclictest0-21swapper/007:31:150
2593036992117,2cyclictest0-21swapper/009:00:110
2593036992116,3cyclictest945-21thermald11:20:040
259303699210,20cyclictest0-21swapper/012:30:470
259303699210,20cyclictest0-21swapper/008:33:550
259303699210,20cyclictest0-21swapper/008:17:250
259303699210,19cyclictest3056-21PK-Backend11:45:250
139099210,14rtkit-daemon3056-21PK-Backend10:55:123
139099210,14rtkit-daemon0-21swapper/312:34:433
2593048992018,1cyclictest3056-21PK-Backend07:45:463
2593048992018,1cyclictest0-21swapper/307:41:343
2593048992013,5cyclictest2647644-21python309:40:213
2593048992012,6cyclictest2618918-21awk08:20:163
259304899201,17cyclictest2681710-21apt11:20:113
259304899200,1cyclictest0-21swapper/309:33:483
259304899200,1cyclictest0-21swapper/308:14:263
259304899200,1cyclictest0-21swapper/308:14:263
259304899200,19cyclictest3056-21PK-Backend09:08:203
259304899200,19cyclictest0-21swapper/312:37:273
259304899200,19cyclictest0-21swapper/312:25:453
259304899200,19cyclictest0-21swapper/311:15:013
259304899200,19cyclictest0-21swapper/308:55:223
259304899200,19cyclictest0-21swapper/308:50:093
259304899200,19cyclictest0-21swapper/308:18:113
259304899200,19cyclictest0-21swapper/307:20:183
259304899200,18cyclictest3056-21PK-Backend07:25:393
259304899200,18cyclictest2678986-21ntp_states11:10:203
2593043992018,1cyclictest3056-21PK-Backend09:21:362
2593043992018,1cyclictest0-21swapper/212:08:052
2593043992018,1cyclictest0-21swapper/210:08:552
2593043992018,1cyclictest0-21swapper/209:48:062
2593043992018,1cyclictest0-21swapper/208:45:062
2593043992018,1cyclictest0-21swapper/208:12:482
2593043992018,1cyclictest0-21swapper/208:12:482
2593043992018,1cyclictest0-21swapper/207:23:552
259304399200,1cyclictest3056-21PK-Backend08:20:192
259304399200,1cyclictest0-21swapper/208:32:092
259304399200,19cyclictest3056-21PK-Backend09:05:272
259304399200,19cyclictest0-21swapper/211:58:392
259304399200,19cyclictest0-21swapper/211:16:142
259304399200,19cyclictest0-21swapper/210:00:022
259304399200,19cyclictest0-21swapper/209:13:252
259304399200,19cyclictest0-21swapper/208:37:562
259304399200,19cyclictest0-21swapper/207:31:522
2593038992018,1cyclictest3056-21PK-Backend11:10:541
2593038992018,1cyclictest3056-21PK-Backend10:45:061
2593038992018,1cyclictest3056-21PK-Backend09:42:471
2593038992018,1cyclictest3056-21PK-Backend09:35:311
2593038992018,1cyclictest3056-21PK-Backend08:35:271
2593038992018,1cyclictest3056-21PK-Backend08:27:571
2593038992014,4cyclictest2156-21gnome-shell12:09:091
2593038992012,6cyclictest2607470-21awk07:45:241
259303899200,19cyclictest3056-21PK-Backend11:23:321
259303899200,19cyclictest3056-21PK-Backend08:19:121
259303899200,19cyclictest0-21swapper/111:01:491
259303899200,19cyclictest0-21swapper/110:35:461
259303899200,19cyclictest0-21swapper/107:11:181
2593036992018,1cyclictest0-21swapper/012:07:490
2593036992018,1cyclictest0-21swapper/012:02:010
2593036992018,1cyclictest0-21swapper/011:25:100
2593036992018,1cyclictest0-21swapper/011:01:170
2593036992018,1cyclictest0-21swapper/010:50:360
2593036992018,1cyclictest0-21swapper/010:45:560
2593036992018,1cyclictest0-21swapper/008:42:130
2593036992018,1cyclictest0-21swapper/008:25:500
2593036992018,1cyclictest0-21swapper/007:58:470
2593036992018,1cyclictest0-21swapper/007:51:090
2593036992018,1cyclictest0-21swapper/007:44:130
259303699200,1cyclictest0-21swapper/010:11:150
259303699200,1cyclictest0-21swapper/008:23:210
259303699200,19cyclictest0-21swapper/011:58:360
259303699200,19cyclictest0-21swapper/011:05:020
259303699200,19cyclictest0-21swapper/010:43:120
259303699200,19cyclictest0-21swapper/010:25:210
259303699200,19cyclictest0-21swapper/009:53:100
27054452190,2sleep00-21swapper/012:25:180
2593048991917,1cyclictest3056-21PK-Backend07:19:303
2593048991917,1cyclictest0-21swapper/311:26:173
2593048991917,1cyclictest0-21swapper/310:30:063
2593048991914,3cyclictest2657157-21basename10:10:113
2593048991914,3cyclictest2636479-21df09:10:133
2593048991914,3cyclictest2620840-21memory08:25:173
2593048991912,5cyclictest2601621-21awk07:30:183
2593048991911,6cyclictest2626583-21python308:40:203
259304899190,1cyclictest3056-21PK-Backend12:00:573
259304899190,1cyclictest3056-21PK-Backend10:52:023
259304899190,1cyclictest0-21swapper/310:08:023
259304899190,18cyclictest3056-21PK-Backend09:03:063
2593043991917,1cyclictest0-21swapper/211:33:382
2593043991917,1cyclictest0-21swapper/209:55:032
2593043991917,1cyclictest0-21swapper/209:02:112
2593043991917,1cyclictest0-21swapper/207:54:442
2593043991917,1cyclictest0-21swapper/207:48:282
2593043991914,3cyclictest2691865-21idleruntime-cro11:50:002
2593043991912,5cyclictest2631067-21cpuspeed_turbos08:55:122
2593043991910,7cyclictest2636-21xdg-desktop-por11:37:042
259304399190,1cyclictest0-21swapper/211:14:382
259304399190,1cyclictest0-21swapper/210:18:352
259304399190,18cyclictest0-21swapper/208:06:552
2593038991912,5cyclictest2700205-21cat12:10:181
2593038991911,6cyclictest2695582-21awk12:00:111
259303899190,1cyclictest3056-21PK-Backend10:17:341
259303899190,1cyclictest3056-21PK-Backend10:13:081
259303899190,1cyclictest3056-21PK-Backend09:14:081
259303899190,1cyclictest3056-21PK-Backend08:52:381
259303899190,1cyclictest0-21swapper/107:17:011
259303899190,18cyclictest0-21swapper/109:21:311
2593036991917,1cyclictest0-21swapper/012:14:240
2593036991917,1cyclictest0-21swapper/010:36:370
2593036991917,1cyclictest0-21swapper/010:31:570
2593036991917,1cyclictest0-21swapper/008:46:400
2593036991913,4cyclictest2600057-21grep07:25:190
2593036991912,5cyclictest2704071-21sed12:20:220
259303699190,1cyclictest0-21swapper/011:33:140
259303699190,1cyclictest0-21swapper/010:04:530
259303699190,1cyclictest0-21swapper/009:26:000
259303699190,1cyclictest0-21swapper/009:15:530
259303699190,1cyclictest0-21swapper/008:58:490
259303699190,1cyclictest0-21swapper/008:12:180
259303699190,1cyclictest0-21swapper/008:12:180
259303699190,18cyclictest0-21swapper/011:12:330
259303699190,18cyclictest0-21swapper/009:21:540
259303699190,18cyclictest0-21swapper/009:07:550
259303699190,18cyclictest0-21swapper/008:35:510
139099190,13rtkit-daemon0-21swapper/310:00:083
259304899182,12cyclictest2692126-21apt11:50:113
2593048991816,1cyclictest0-21swapper/312:15:583
2593048991816,1cyclictest0-21swapper/308:39:543
2593048991811,6cyclictest2691319-21sed11:45:223
2593048991811,5cyclictest2666850-21ntp_states10:35:223
259304899180,16cyclictest436-20systemd-journal10:44:593
259304899180,16cyclictest2639844-21apt09:20:123
259304399189,7cyclictest883-21dbus-daemon07:45:012
259304399187,8cyclictest2639544-21sort09:20:002
2593043991816,1cyclictest0-21swapper/210:38:352
2593043991816,1cyclictest0-21swapper/210:14:432
2593038991816,1cyclictest0-21swapper/108:30:271
2593038991816,1cyclictest0-21swapper/107:30:041
2593038991815,2cyclictest3056-21PK-Backend11:33:121
2593038991815,2cyclictest3056-21PK-Backend10:29:101
2593038991812,4cyclictest2156-21gnome-shell10:52:091
2593038991811,6cyclictest2673963-21cat10:55:231
2593036991816,1cyclictest0-21swapper/009:42:020
2593036991816,1cyclictest0-21swapper/007:21:560
2593036991816,1cyclictest0-21swapper/007:15:130
2593048991715,1cyclictest3056-21PK-Backend09:27:133
259304899171,14cyclictest2648486-21/usr/sbin/munin09:45:103
2593048991710,5cyclictest2689381-21ntp_kernel_pll_11:40:193
259304899170,15cyclictest2659280-21diskstats10:15:143
259304899170,15cyclictest2638582-21ntp_states09:15:183
259304899170,15cyclictest2604389-21cron07:40:003
2593043991715,1cyclictest0-21swapper/207:39:172
259303899179,6cyclictest2677462-21expr11:05:221
2593038991715,1cyclictest3056-21PK-Backend11:36:091
2593038991715,1cyclictest3056-21PK-Backend11:27:171
2593038991715,1cyclictest0-21swapper/109:03:401
2593038991711,5cyclictest2694332-21awk11:55:161
2593036991715,1cyclictest0-21swapper/009:58:450
2593036991715,1cyclictest0-21swapper/007:47:380
2593048991614,1cyclictest3056-21PK-Backend11:02:103
2593048991614,1cyclictest3056-21PK-Backend07:13:363
259303899169,5cyclictest2610150-21cat07:55:151
2593038991614,1cyclictest3056-21PK-Backend08:03:171
2593038991614,1cyclictest3056-21PK-Backend07:44:381
2593038991614,1cyclictest0-21swapper/108:55:561
2593038991613,2cyclictest3056-21PK-Backend10:05:051
2593038991611,3cyclictest2703231-21tr12:20:121
259303699168,7cyclictest2644783-21basename09:35:000
2593036991614,1cyclictest0-21swapper/012:19:250
2593036991614,1cyclictest0-21swapper/010:23:250
2593036991610,4cyclictest2593636-21grep07:10:120
2593048991513,1cyclictest0-21swapper/311:55:093
2593038991511,3cyclictest1095-21runrttasks11:54:191
259303899151,11cyclictest2618503-21/usr/sbin/munin08:20:231
2593036991513,1cyclictest0-21swapper/008:06:370
259303899149,3cyclictest2638353-21cut09:15:161
2593038991412,1cyclictest3056-21PK-Backend09:50:511
259303699147,5cyclictest2655644-21cat10:05:120
2593036991412,1cyclictest0-21swapper/009:47:030
2593043991311,1cyclictest0-21swapper/209:25:082
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional