You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-25 - 05:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Tue Aug 25, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
83421310,0sleep30-21swapper/300:05:253
921629481,9sleep20-21swapper/219:06:192
934429279,9sleep00-21swapper/019:07:580
927528769,13sleep10-21swapper/119:07:081
932228067,9sleep30-21swapper/319:07:413
260082500,1sleep30-21swapper/323:00:413
9589992824,2cyclictest23157-21df_abs00:20:111
958999250,25cyclictest0-21swapper/123:28:151
9587992222,0cyclictest0-21swapper/000:26:500
959299210,21cyclictest0-21swapper/221:50:242
9587992119,0cyclictest0-21swapper/022:21:070
958799200,20cyclictest0-21swapper/021:38:570
958799200,20cyclictest0-21swapper/021:38:570
9592991918,1cyclictest0-21swapper/222:59:342
959299190,19cyclictest0-21swapper/222:01:532
958999197,10cyclictest38150irq/131-snd_hda00:09:501
9589991917,1cyclictest4901-21sed20:05:201
9587991919,0cyclictest0-21swapper/021:28:100
959299182,2cyclictest141rcu_preempt22:35:342
959299180,1cyclictest55750irq/128-enp2s0-21:23:032
958999180,1cyclictest0-21swapper/121:44:181
958799180,1cyclictest0-21swapper/021:43:060
958799180,17cyclictest0-21swapper/021:10:130
35702180,2sleep232-21ksoftirqd/221:30:002
9596991715,1cyclictest39-21ksoftirqd/323:10:213
9596991715,1cyclictest39-21ksoftirqd/300:30:313
9592991716,1cyclictest0-21swapper/223:05:232
959299170,17cyclictest25841-21ssh23:00:332
959299170,0cyclictest0-21swapper/222:48:432
958799176,6cyclictest32316-21perf23:25:000
958799176,6cyclictest32316-21perf23:25:000
9592991615,1cyclictest0-21swapper/222:25:132
9592991614,1cyclictest0-21swapper/222:23:432
9592991614,1cyclictest0-21swapper/220:30:142
959299160,1cyclictest0-21swapper/200:06:472
9589991613,3cyclictest0-21swapper/123:40:221
958999160,15cyclictest0-21swapper/123:15:181
958999160,0cyclictest0-21swapper/123:09:201
958799164,6cyclictest23836-21munin-run19:40:000
958799164,6cyclictest23836-21munin-run19:40:000
958799160,9cyclictest0-21swapper/021:53:560
9592991514,1cyclictest0-21swapper/222:30:042
9592991514,1cyclictest0-21swapper/222:06:532
959299150,15cyclictest0-21swapper/223:36:422
9589991513,2cyclictest0-21swapper/122:00:001
9589991512,2cyclictest0-21swapper/120:15:161
958999150,1cyclictest0-21swapper/121:11:281
958999150,14cyclictest0-21swapper/100:26:161
958999150,13cyclictest26618-21cat19:45:001
958799153,6cyclictest0-21swapper/023:25:130
959699144,5cyclictest39-21ksoftirqd/321:59:113
9596991413,1cyclictest0-21swapper/319:40:143
9596991411,2cyclictest32287-21cron23:25:013
9596991411,2cyclictest32287-21cron23:25:003
959699140,13cyclictest12622-21grep20:20:263
959699140,11cyclictest0-21swapper/323:51:473
959299149,3cyclictest20228-21cron20:40:002
959299145,6cyclictest0-21swapper/221:11:272
959299144,8cyclictest0-21swapper/200:00:252
9592991414,0cyclictest0-21swapper/222:50:432
9592991413,1cyclictest0-21swapper/221:15:232
9592991411,2cyclictest24721-21tr23:20:142
9592991411,2cyclictest24721-21tr23:20:142
9592991410,2cyclictest21656-21cron22:20:002
958999148,2cyclictest0-21swapper/122:05:151
9589991410,3cyclictest29362-21tr19:50:141
958999140,13cyclictest0-21swapper/120:45:151
9587991412,2cyclictest13-21ksoftirqd/022:39:200
958799140,1cyclictest0-21swapper/023:36:590
958799140,1cyclictest0-21swapper/019:30:130
958799140,11cyclictest0-21swapper/022:17:530
958799140,0cyclictest0-21swapper/000:20:420
959699139,3cyclictest7074-21ls22:10:203
959699139,3cyclictest13921-21basename22:15:133
959699135,6cyclictest17323-21apt-get23:35:243
959699134,4cyclictest21206-21/usr/sbin/munin21:20:233
959699132,10cyclictest55850irq/129-enp2s0-23:17:383
959699131,1cyclictest141rcu_preempt21:15:153
9596991311,1cyclictest39-21ksoftirqd/322:41:553
9596991310,2cyclictest21117-21rm21:40:153
9596991310,2cyclictest18204-21grep20:35:153
9596991310,2cyclictest11033-21ls19:10:213
959699130,12cyclictest0-21swapper/322:31:303
959299139,3cyclictest1731-21sed23:25:212
959299139,2cyclictest21439-21cron19:35:002
959299139,2cyclictest20450-21cat21:40:002
959299139,2cyclictest20450-21cat21:40:002
959299133,5cyclictest225-21systemd-journal19:40:002
959299133,5cyclictest225-21systemd-journal19:40:002
9592991312,1cyclictest55850irq/129-enp2s0-00:34:032
9592991312,1cyclictest0-21swapper/221:48:132
9592991310,2cyclictest9442-21sed23:30:172
9592991310,2cyclictest31364-21cat19:55:002
9592991310,2cyclictest3099-21grep20:00:272
9592991310,2cyclictest12489-21grep19:15:162
959299130,1cyclictest0-21swapper/221:00:212
958999139,3cyclictest30515-21tr21:00:151
958999139,3cyclictest20239-21cron21:40:001
958999139,3cyclictest20239-21cron21:40:001
958999138,4cyclictest4624-21ssh22:48:231
9589991310,3cyclictest25-21ksoftirqd/122:14:031
9589991310,2cyclictest15588-21grep00:15:001
9589991310,2cyclictest13122-21turbostat20:24:591
958999130,12cyclictest3534-21ssh21:30:001
958999130,12cyclictest0-21swapper/120:30:231
958799139,3cyclictest11605-21sort21:15:170
958799138,3cyclictest7228-21ssh22:30:170
958799131,7cyclictest15510-21perf20:30:000
9587991310,2cyclictest9576-21sort23:10:250
9587991310,2cyclictest7637-21grep22:10:250
9587991310,2cyclictest7361-21cat00:10:000
9587991310,2cyclictest28534-21grep20:55:160
9587991310,2cyclictest24766-21sed22:40:260
9587991310,2cyclictest15699-21grep22:55:150
958799130,8cyclictest19092-21awk19:30:000
958799130,7cyclictest22651-21awk20:45:010
958799130,12cyclictest12112-21grep20:20:200
959699129,2cyclictest6551-21ssh21:50:233
959699129,2cyclictest32724-21grep21:05:133
959699129,2cyclictest31603-21ssh22:45:153
959699129,2cyclictest29670-21grep19:50:163
959699129,2cyclictest28349-21df20:55:143
959699129,2cyclictest27451-21head19:45:173
959699129,2cyclictest24311-21sed20:45:223
959699129,2cyclictest23368-21sed22:00:273
959699129,2cyclictest20948-21grep19:30:243
959699129,2cyclictest15948-21sed00:35:163
959699129,2cyclictest14012-21cut20:25:183
959699129,2cyclictest11891-21sleep21:35:003
959699128,3cyclictest30971-21head21:00:173
959699127,4cyclictest19045-21unixbench_multi23:57:193
959699120,2cyclictest0-21swapper/321:37:533
959699120,2cyclictest0-21swapper/321:37:533
959699120,1cyclictest0-21swapper/322:27:133
959299129,3cyclictest0-21swapper/221:55:132
959299129,2cyclictest9819-21sed23:10:262
959299129,2cyclictest7851-21grep20:10:262
959299129,2cyclictest6502-21grep22:10:152
959299129,2cyclictest32-21ksoftirqd/220:45:202
959299129,2cyclictest19095-21ssh23:57:182
959299128,3cyclictest18628-21grep19:25:302
959299123,9cyclictest0-21swapper/200:39:382
9592991211,1cyclictest55750irq/128-enp2s0-22:40:432
959299120,11cyclictest22387-21sh21:40:242
958999129,2cyclictest8278-21grep22:30:261
958999129,2cyclictest38150irq/131-snd_hda22:23:471
958999129,2cyclictest26390-21head23:20:251
958999129,2cyclictest26390-21head23:20:251
958999129,2cyclictest22864-21basename22:40:131
958999129,2cyclictest18741-21wc23:35:331
958999129,2cyclictest15765-21grep22:55:151
958999129,2cyclictest15599-21cut23:55:131
958999129,2cyclictest14138-21head20:25:181
958999129,2cyclictest12798-21grep19:15:171
958999129,1cyclictest23563-21cpuspeed_turbos00:00:121
958999128,3cyclictest30636-21wc21:45:251
958999128,3cyclictest21461-21basename19:35:011
958999128,1cyclictest626-21ssh22:25:511
958999127,3cyclictest7978-21cut22:50:161
9589991212,0cyclictest0-21swapper/121:54:221
958999120,12cyclictest0-21swapper/123:47:571
958999120,12cyclictest0-21swapper/121:15:161
958999120,11cyclictest32215-21tail19:55:181
958799129,2cyclictest7008-21chrt00:30:150
958799129,2cyclictest5228-21munin-plugin-st22:10:000
958799129,2cyclictest25612-21sed23:00:280
958799129,2cyclictest2367-21cron21:10:000
958799129,2cyclictest21755-21tr22:00:150
958799129,2cyclictest19974-21tr21:20:160
958799129,2cyclictest15655-21sort00:15:000
958799129,2cyclictest13448-21cut21:55:150
958799128,3cyclictest3586-21cron20:05:000
958799124,4cyclictest225-21systemd-journal20:50:000
958799124,4cyclictest16306-21aten2_r7power_p23:15:150
958799121,1cyclictest0-21swapper/021:30:150
9587991211,1cyclictest0-21swapper/023:07:590
958799120,2cyclictest0-21swapper/000:38:350
958799120,11cyclictest0-21swapper/020:50:130
959699118,2cyclictest6453-21sh23:28:423
959699118,2cyclictest5010-21cat20:05:213
959699118,2cyclictest3823-21ssh23:07:123
959699118,2cyclictest32567-21ssh22:05:573
959699118,2cyclictest31352-21ssh23:43:363
959699118,2cyclictest30808-21ssh00:25:093
959699118,2cyclictest2859-21ssh21:48:453
959699118,2cyclictest25474-21ssh22:21:283
959699118,2cyclictest25006-21ssh00:21:073
959699118,2cyclictest23539-21ssh00:00:123
959699118,2cyclictest17767-21ssh00:16:343
959699118,2cyclictest1704-21grep20:00:153
959699118,2cyclictest15953-21head22:35:203
959699118,2cyclictest15460-21ssh23:34:173
959699118,2cyclictest12915-21head19:15:183
959699118,2cyclictest12321-21ssh00:12:263
959699118,2cyclictest11742sleep323:45:013
959699118,2cyclictest10013-21ssh22:51:083
959699117,3cyclictest3034-21grep21:10:143
959699115,2cyclictest225-21systemd-journal20:45:003
959699113,7cyclictest39-21ksoftirqd/320:35:003
959299118,2cyclictest955-21ssh00:25:482
959299118,2cyclictest8138-21tr00:10:172
959299118,2cyclictest5693-21ssh21:30:262
959299118,2cyclictest28988-21ntpq20:55:192
959299118,2cyclictest2695-21ssh23:46:132
959299118,2cyclictest25215-21ssh23:40:052
959299118,2cyclictest2503-21ssh21:10:002
959299118,2cyclictest19246-21ssh00:17:442
959299118,2cyclictest13379-21sh23:54:182
959299118,2cyclictest10263-21cat19:10:142
959299117,3cyclictest28991-21wc19:50:002
959299110,10cyclictest21915-21sensors_temp20:40:242
959299110,10cyclictest16469-21sh23:15:152
958999119,1cyclictest14892-21tr22:35:131
958999118,2cyclictest7196-21wc20:10:191
958999118,2cyclictest5970-21sed21:30:281
958999118,2cyclictest29955-21cron21:00:001
958999118,2cyclictest26274-21head20:50:171
958999118,2cyclictest2622-21ntpq20:00:211
958999118,2cyclictest23652-21ssh23:00:111
958999118,2cyclictest22778-21cut20:45:011
958999118,2cyclictest21785-21ssh21:20:301
958999118,2cyclictest19772-21sed20:35:271
958999118,2cyclictest17209-21ssh00:16:071
958999118,2cyclictest15916-21head00:35:161
958999118,2cyclictest11934-21ls19:15:001
958999118,2cyclictest10478-21sed23:30:261
958999117,3cyclictest9113-21ssh23:50:571
958999117,3cyclictest2395-21basename21:10:011
958999117,3cyclictest11112-21ssh00:32:151
958999111,9cyclictest2391-21diskmemload22:00:161
9589991110,1cyclictest25-21ksoftirqd/123:10:151
958799119,1cyclictest24210-21head00:00:160
958799118,2cyclictest9268-21ssh23:51:040
958799118,2cyclictest6859-21grep20:10:170
958799118,2cyclictest3667-21ssh22:28:160
958799118,2cyclictest3496-21ssh23:46:520
958799118,2cyclictest32378-21taskset21:04:350
958799118,2cyclictest31519-21grep22:45:150
958799118,2cyclictest29823-21grep21:45:200
958799118,2cyclictest29676-21grep19:50:170
958799118,2cyclictest27052-21ls19:45:150
958799118,2cyclictest26431-21sh19:45:000
958799118,2cyclictest25446-21basename23:40:150
958799118,2cyclictest20932-21ssh23:58:420
958799118,2cyclictest16294-21ssh00:15:240
958799118,2cyclictest12337-21grep19:15:150
958799118,2cyclictest11162-21ssh23:30:530
958799118,2cyclictest10549-21ssh22:51:330
959699108,1cyclictest6726-21grep20:10:163
959699108,1cyclictest30101-21chrt21:25:283
959699108,1cyclictest27341-21taskset20:51:553
959699108,1cyclictest16400-21chrt19:21:153
959699108,1cyclictest15322-21ssh22:55:083
959699108,1cyclictest10083-21cat20:15:273
959699107,2cyclictest22434-21grep19:35:183
959699107,2cyclictest22434-21grep19:35:173
959699103,3cyclictest16980-21apt-get19:25:183
959299108,1cyclictest4166-21cat20:05:152
959299108,1cyclictest25942-21expr20:50:162
959299108,1cyclictest22402-21ssh00:20:022
959299108,1cyclictest15170-21chrt20:26:022
959299108,1cyclictest10522-21taskset20:17:312
959299107,2cyclictest14649-21cut19:20:142
959299107,2cyclictest11435-21expr20:20:152
959299100,9cyclictest24196-21gpg-connect-age19:40:012
958999108,1cyclictest18804-21taskset19:26:401
958999108,1cyclictest14881-21expr19:20:151
958999107,2cyclictest13924-21ssh22:15:121
958999104,3cyclictest23229-21timerandwakeup19:35:251
958999104,3cyclictest23229-21timerandwakeup19:35:251
958799108,1cyclictest5829-21taskset20:08:210
958799108,1cyclictest20012-21taskset20:37:140
958799107,2cyclictest9389-21head20:15:190
958799107,2cyclictest17265-21wc20:30:270
958799107,2cyclictest10393-21df_inode19:10:150
958799106,3cyclictest32611-21head19:55:210
958799101,1cyclictest0-21swapper/019:20:210
95969996,2cyclictest31688-21tr19:55:163
95929996,2cyclictest32559-21wc19:55:212
95899997,1cyclictest28076-21cut19:45:231
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional