You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-20 - 22:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Mon Jul 20, 2026 12:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
305487723010,9sleep20-21swapper/207:05:152
305583128340,14sleep10-21swapper/107:07:361
305454927744,23sleep30-21swapper/307:05:123
305598627638,9sleep00-21swapper/007:09:360
31540102500,3sleep20-21swapper/211:45:012
31275652420,6sleep10-21swapper/110:30:011
139099280,4rtkit-daemon1389-21rtkit-daemon11:58:302
305615599272,21cyclictest3126478-21python310:25:223
31291562260,3sleep33056-21PK-Backend10:35:003
305615599268,16cyclictest3104714-21basename09:25:163
3056155992614,10cyclictest3056-21PK-Backend11:57:453
305615599261,23cyclictest3148058-21python311:25:193
3056155992610,14cyclictest3114810-21sort09:55:133
305615599260,7cyclictest3108419-21grep09:35:183
305615599260,24cyclictest3163802-21ntp_states12:10:193
305615599260,24cyclictest3149301-21df11:30:153
139099260,4rtkit-daemon1389-21rtkit-daemon09:10:562
305615599256,17cyclictest3165193-21sh12:15:163
305615599252,6cyclictest3136367-21cat10:55:013
305615599251,5cyclictest3114453-21munin-run09:55:013
305615599251,22cyclictest3146917-21cron11:25:003
305615599250,5cyclictest3162614-21sed12:10:003
305615599250,5cyclictest3059824-21switchtime07:15:233
305615599246,16cyclictest3062271-21cat07:25:013
305615599243,19cyclictest0-21swapper/311:45:163
305615599242,19cyclictest3124755-21/usr/sbin/munin10:20:223
3056155992419,4cyclictest0-21swapper/307:45:033
305615599241,3cyclictest3136882-21cstates10:55:133
3056155992411,11cyclictest3151336-21strings11:35:183
3056155992411,11cyclictest3151336-21strings11:35:183
305615599240,4cyclictest31343412sleep310:47:483
305615599240,19cyclictest3056-21PK-Backend08:07:273
3056146992418,4cyclictest3118129-21mktemp10:05:000
305615599238,13cyclictest42-21ksoftirqd/311:45:003
305615599232,17cyclictest3085231-21apt08:30:123
305615599232,17cyclictest0-21swapper/307:10:113
3056155992320,2cyclictest3056-21PK-Backend08:46:263
305615599231,6cyclictest3141742-21munin-plugin-st11:10:013
305615599231,3cyclictest0-21swapper/308:35:133
305614999237,14cyclictest436-20systemd-journal10:20:001
3056149992318,3cyclictest3058020-40unattended-upgr07:13:441
3056149992316,5cyclictest3137988-21cron11:00:001
305614999230,21cyclictest3069746-21df_inode07:45:131
3056146992318,3cyclictest3119115-21netstat10:05:170
30889042220,4chrt0-21swapper/008:40:120
305615599228,12cyclictest3056-21PK-Backend07:40:213
305615599226,14cyclictest3106043-21cpu09:30:123
3056155992220,1cyclictest3056-21PK-Backend07:55:433
3056155992218,3cyclictest3056-21PK-Backend09:42:513
3056155992218,3cyclictest3056-21PK-Backend08:24:093
305615599221,3cyclictest0-21swapper/308:25:223
305615599221,3cyclictest0-21swapper/307:25:093
305615599220,20cyclictest3056-21PK-Backend10:19:293
305615599220,20cyclictest3056-21PK-Backend09:12:563
305615599220,20cyclictest3056-21PK-Backend08:54:053
3056154992220,1cyclictest0-21swapper/211:02:352
3056154992220,1cyclictest0-21swapper/207:33:272
3056154992220,1cyclictest0-21swapper/207:33:262
305615499220,3cyclictest3092480-21expr08:50:122
3056149992214,6cyclictest3155917-21awk11:50:131
305614699222,19cyclictest0-21swapper/012:22:400
3056146992214,6cyclictest436-20systemd-journal08:10:010
31249552210,4sleep10-21swapper/110:20:221
305615599216,13cyclictest3169734-21latency_hist12:30:013
3056155992119,1cyclictest3056-21PK-Backend12:30:353
3056155992119,1cyclictest3056-21PK-Backend12:21:363
3056155992119,1cyclictest3056-21PK-Backend11:15:193
3056155992119,1cyclictest3056-21PK-Backend07:53:003
3056155992111,9cyclictest3056-21PK-Backend11:50:343
305615599210,20cyclictest3056-21PK-Backend10:13:203
3056154992119,1cyclictest0-21swapper/212:33:062
3056154992119,1cyclictest0-21swapper/212:13:052
3056154992119,1cyclictest0-21swapper/212:06:092
3056154992119,1cyclictest0-21swapper/212:01:362
3056154992119,1cyclictest0-21swapper/211:37:012
3056154992119,1cyclictest0-21swapper/211:37:012
3056154992119,1cyclictest0-21swapper/211:12:272
3056154992119,1cyclictest0-21swapper/210:36:042
3056154992119,1cyclictest0-21swapper/209:51:432
3056154992119,1cyclictest0-21swapper/209:25:422
3056154992119,1cyclictest0-21swapper/209:07:572
3056154992119,1cyclictest0-21swapper/207:53:082
3056154992119,1cyclictest0-21swapper/207:15:292
3056154992118,2cyclictest3056-21PK-Backend09:20:132
3056154992118,2cyclictest0-21swapper/208:15:252
3056154992114,6cyclictest3147587-21forks11:25:152
3056154992112,2cyclictest34-21ksoftirqd/209:45:212
305615499210,20cyclictest0-21swapper/211:30:532
305615499210,20cyclictest0-21swapper/211:17:012
305615499210,20cyclictest0-21swapper/210:54:512
305615499210,1cyclictest0-21swapper/210:12:452
3056149992116,3cyclictest3102277-21cron09:20:011
3056149992114,5cyclictest3170049-21chmod12:30:001
305614999210,20cyclictest3056-21PK-Backend09:32:541
305614999210,20cyclictest0-21swapper/111:55:591
305614999210,20cyclictest0-21swapper/109:02:391
3056146992119,1cyclictest0-21swapper/010:18:230
3056146992112,7cyclictest3074302-21apt-get08:00:010
305614699210,20cyclictest0-21swapper/011:18:120
305615599204,4cyclictest3131534-21ls10:40:143
3056155992018,1cyclictest3056-21PK-Backend09:20:483
3056155992018,1cyclictest3056-21PK-Backend09:09:593
3056155992018,1cyclictest3056-21PK-Backend09:00:333
3056155992018,1cyclictest3056-21PK-Backend08:13:083
3056155992018,1cyclictest3056-21PK-Backend07:38:003
3056155992013,5cyclictest3173470-21apt12:39:593
3056155992013,5cyclictest3173470-21apt12:39:593
305615599200,1cyclictest3056-21PK-Backend10:35:363
305615599200,1cyclictest3056-21PK-Backend09:15:073
305615599200,1cyclictest3056-21PK-Backend08:15:173
305615499207,2cyclictest141rcu_preempt10:21:202
3056154992018,1cyclictest0-21swapper/211:23:102
3056154992018,1cyclictest0-21swapper/210:30:442
3056154992018,1cyclictest0-21swapper/208:20:192
3056154992018,1cyclictest0-21swapper/207:45:372
3056154992012,6cyclictest3115688-21sed09:55:222
3056154992012,6cyclictest3106022-21cat09:30:112
305615499200,5cyclictest0-21swapper/210:05:202
305615499200,19cyclictest0-21swapper/212:26:572
305615499200,19cyclictest0-21swapper/211:50:542
305615499200,19cyclictest0-21swapper/211:45:342
305615499200,19cyclictest0-21swapper/210:28:212
3056149992018,1cyclictest0-21swapper/111:35:031
3056149992018,1cyclictest0-21swapper/111:35:021
3056149992018,1cyclictest0-21swapper/111:06:581
3056149992018,1cyclictest0-21swapper/110:47:371
3056149992018,1cyclictest0-21swapper/108:54:021
3056149992018,1cyclictest0-21swapper/107:43:241
3056149992017,2cyclictest3056-21PK-Backend07:16:401
3056149992015,4cyclictest0-21swapper/112:20:181
3056149992013,6cyclictest3148061-21python311:25:211
3056149992013,4cyclictest3121617-21latency_hist10:15:011
305614999200,19cyclictest3056-21PK-Backend09:53:361
305614999200,19cyclictest3056-21PK-Backend08:37:251
305614999200,19cyclictest3056-21PK-Backend08:32:041
305614999200,19cyclictest0-21swapper/112:32:281
305614999200,19cyclictest0-21swapper/111:47:001
305614999200,19cyclictest0-21swapper/111:30:171
305614999200,19cyclictest0-21swapper/110:37:041
305614999200,19cyclictest0-21swapper/110:08:511
305614999200,19cyclictest0-21swapper/109:21:331
305614999200,19cyclictest0-21swapper/108:42:531
305614999200,19cyclictest0-21swapper/107:35:341
3056146992018,1cyclictest0-21swapper/011:46:430
3056146992018,1cyclictest0-21swapper/008:50:490
3056146992018,1cyclictest0-21swapper/007:19:170
3056146992018,1cyclictest0-21swapper/007:11:120
3056146992013,5cyclictest3072720-21cut07:55:000
3056146992012,6cyclictest3168249-21find12:25:120
3056146992012,6cyclictest3158463-21python311:55:230
3056146992012,6cyclictest3079766-21ls08:15:000
305614699200,1cyclictest0-21swapper/009:29:430
305614699200,19cyclictest3056-21PK-Backend09:30:290
305614699200,19cyclictest0-21swapper/012:12:530
305614699200,19cyclictest0-21swapper/011:27:100
305614699200,18cyclictest3056-21PK-Backend10:30:180
139099200,3rtkit-daemon1389-21rtkit-daemon07:40:092
3056155991916,2cyclictest3056-21PK-Backend10:02:383
305615599190,18cyclictest3056-21PK-Backend09:45:493
305615599190,18cyclictest3056-21PK-Backend08:59:463
305615599190,17cyclictest3056-21PK-Backend10:05:163
305615499192,16cyclictest3102294-21idleruntime-cro09:20:002
3056154991917,1cyclictest0-21swapper/209:35:282
3056154991917,1cyclictest0-21swapper/208:48:172
3056154991917,1cyclictest0-21swapper/208:12:412
3056154991917,1cyclictest0-21swapper/207:14:412
3056154991913,4cyclictest3066516-21grep07:35:172
3056154991912,5cyclictest3094009-21sshd08:55:122
305615499190,1cyclictest0-21swapper/212:19:132
3056149991917,1cyclictest3056-21PK-Backend12:07:201
3056149991917,1cyclictest0-21swapper/111:19:071
3056149991917,1cyclictest0-21swapper/110:02:421
3056149991917,1cyclictest0-21swapper/109:46:551
3056149991917,1cyclictest0-21swapper/109:39:241
3056149991917,1cyclictest0-21swapper/108:21:371
3056149991917,1cyclictest0-21swapper/108:14:551
3056149991917,1cyclictest0-21swapper/108:07:031
3056149991917,1cyclictest0-21swapper/108:03:331
3056149991917,1cyclictest0-21swapper/107:57:171
305614999191,5cyclictest141rcu_preempt09:25:201
3056149991911,3cyclictest26-21ksoftirqd/107:30:121
3056149991911,3cyclictest26-21ksoftirqd/107:30:121
305614999190,1cyclictest3056-21PK-Backend11:23:541
305614999190,1cyclictest3056-21PK-Backend07:22:351
305614999190,1cyclictest0-21swapper/109:05:381
305614999190,1cyclictest0-21swapper/108:56:111
305614999190,1cyclictest0-21swapper/108:25:231
3056146991917,1cyclictest0-21swapper/012:36:390
3056146991917,1cyclictest0-21swapper/012:36:390
3056146991917,1cyclictest0-21swapper/009:40:090
3056146991917,1cyclictest0-21swapper/009:36:240
3056146991917,1cyclictest0-21swapper/008:56:300
3056146991917,1cyclictest0-21swapper/008:16:210
3056146991917,1cyclictest0-21swapper/008:01:280
3056146991917,1cyclictest0-21swapper/007:49:450
3056146991917,1cyclictest0-21swapper/007:38:020
3056146991916,2cyclictest0-21swapper/012:00:430
3056146991914,4cyclictest13-21ksoftirqd/008:25:010
3056146991912,5cyclictest3132329-21sed10:40:230
305614699190,1cyclictest3056-21PK-Backend11:36:570
305614699190,1cyclictest3056-21PK-Backend11:36:570
305614699190,1cyclictest3056-21PK-Backend10:55:260
305614699190,1cyclictest3056-21PK-Backend07:20:040
305614699190,1cyclictest0-21swapper/009:56:460
305614699190,1cyclictest0-21swapper/007:43:160
305614699190,18cyclictest0-21swapper/009:17:330
305614699190,18cyclictest0-21swapper/009:11:030
305614699190,18cyclictest0-21swapper/008:31:220
305614699190,18cyclictest0-21swapper/007:31:130
305614699190,18cyclictest0-21swapper/007:31:120
305615599186,11cyclictest3056-21PK-Backend07:32:523
305615599186,11cyclictest3056-21PK-Backend07:32:513
305615599180,7cyclictest3089410-21awk08:40:183
305615499183,6cyclictest3167600-21users12:20:212
3056154991816,1cyclictest0-21swapper/212:39:142
3056154991816,1cyclictest0-21swapper/212:39:132
3056154991816,1cyclictest0-21swapper/210:47:202
3056154991816,1cyclictest0-21swapper/210:42:472
3056154991816,1cyclictest0-21swapper/209:43:532
3056154991816,1cyclictest0-21swapper/208:43:032
3056154991816,1cyclictest0-21swapper/208:02:472
3056154991816,1cyclictest0-21swapper/207:59:292
3056154991813,3cyclictest34-21ksoftirqd/211:05:132
3056154991811,5cyclictest3084212-21ntpq08:25:182
305614999189,6cyclictest436-20systemd-journal09:15:011
3056149991816,1cyclictest0-21swapper/111:41:241
3056149991812,5cyclictest3165327-21sed12:15:181
3056149991812,4cyclictest3063843-21idleruntime-cro07:30:001
305614999180,1cyclictest0-21swapper/107:51:501
3056146991816,1cyclictest3056-21PK-Backend10:20:460
3056146991816,1cyclictest0-21swapper/009:46:530
3056146991816,1cyclictest0-21swapper/009:08:050
3056146991811,6cyclictest13-21ksoftirqd/010:55:000
3056146991811,5cyclictest3139949-21date11:05:000
3056146991811,5cyclictest3120795-21grep10:10:180
3056146991811,5cyclictest3063870-21systemd-run07:29:590
3056146991810,6cyclictest3164788-21awk12:15:120
31424582170,3chrt3056-21PK-Backend11:10:173
31223482170,3chrt0-21swapper/210:15:102
305615599170,5cyclictest3160796-21turbostat12:05:013
305615599170,5cyclictest3139758-21sleep11:05:003
305615599170,3cyclictest3074774-21cnf-update-db08:00:033
3056154991715,1cyclictest0-21swapper/208:09:582
3056154991712,3cyclictest3137020-21kernelversion10:55:182
3056154991712,3cyclictest3086008-21proc_pri08:30:192
3056154991711,5cyclictest3063842-21cron07:30:002
305614999178,7cyclictest883-21dbus-daemon10:45:011
305614999174,11cyclictest3172869-21python312:35:211
305614999174,11cyclictest3172869-21python312:35:201
305614999172,11cyclictest3090432-21apt08:45:141
3056149991715,1cyclictest0-21swapper/110:25:061
3056149991715,1cyclictest0-21swapper/109:43:091
3056146991715,1cyclictest0-21swapper/011:54:550
3056146991715,1cyclictest0-21swapper/009:21:520
3056146991712,3cyclictest3141107-21processes11:05:210
3056146991712,3cyclictest1095-21runrttasks12:30:000
3056146991711,4cyclictest3097115-21basename09:05:010
305614699170,5cyclictest3056-21PK-Backend08:35:130
3056154991614,1cyclictest0-21swapper/210:00:362
3056154991612,2cyclictest34-21ksoftirqd/207:20:112
3056154991610,4cyclictest34-21ksoftirqd/208:35:122
305614999169,5cyclictest436-20systemd-journal08:19:561
305614999169,5cyclictest3142188-21grep11:10:131
305614999169,4cyclictest3139745-21latency_hist11:05:021
305614699169,5cyclictest3149057-21sort11:30:150
305614699169,5cyclictest3092024-21date08:50:000
305614699169,5cyclictest3083899-21which08:25:160
3056146991614,1cyclictest0-21swapper/010:25:320
3056146991611,3cyclictest3152776-21entropy11:40:140
3056154991511,2cyclictest34-21ksoftirqd/209:00:192
305614999157,6cyclictest3134864-21cat10:50:121
305614999155,8cyclictest0-21swapper/112:00:121
3056149991512,2cyclictest3056-21PK-Backend09:55:121
305614999150,5cyclictest0-21swapper/112:10:131
305614699159,5cyclictest3142162-21expr11:10:140
305614699158,5cyclictest3129426-21dpkg10:35:010
3056146991510,3cyclictest3145661-21cstates11:20:120
3056146991510,3cyclictest3133230-21grep10:45:130
3056146991510,3cyclictest3113151-21cstates09:50:130
305614699148,4cyclictest3161054-21awk12:05:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional