You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-05 - 11:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Wed Aug 05, 2026 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
967729683,9sleep00-21swapper/019:07:130
769629583,8sleep20-21swapper/219:05:012
981528471,9sleep30-21swapper/319:09:003
969928366,13sleep10-21swapper/119:07:291
248202450,0sleep20-21swapper/222:00:562
248202450,0sleep20-21swapper/222:00:552
997999320,31cyclictest0-21swapper/021:28:590
9987992422,2cyclictest0-21swapper/322:00:283
9987992422,2cyclictest0-21swapper/322:00:283
997999241,5cyclictest2759-21diskmemload22:37:050
9987992321,1cyclictest21635-21sh22:00:003
9987992321,1cyclictest21635-21sh21:59:593
998599221,15cyclictest0-21swapper/200:05:152
998599221,10cyclictest0-21swapper/223:20:132
998199220,19cyclictest2759-21diskmemload23:22:581
997999224,4cyclictest4794-21cron21:49:590
998799217,3cyclictest0-21swapper/322:25:423
998199210,2cyclictest0-21swapper/100:16:541
998199210,21cyclictest0-21swapper/121:45:241
997999215,13cyclictest0-21swapper/021:12:140
997999210,3cyclictest0-21swapper/022:49:090
998799209,9cyclictest0-21swapper/323:40:003
998799203,15cyclictest39-21ksoftirqd/320:50:163
9987992018,1cyclictest2759-21diskmemload21:37:393
998799200,13cyclictest39-21ksoftirqd/322:51:173
997999202,4cyclictest12707-21ssh00:10:010
997999200,4cyclictest0-21swapper/000:39:520
9987991917,2cyclictest2759-21diskmemload00:17:343
997999191,4cyclictest2759-21diskmemload00:30:370
998799186,11cyclictest0-21swapper/323:15:493
9987991814,4cyclictest0-21swapper/321:10:033
9985991816,2cyclictest0-21swapper/223:58:592
997999183,7cyclictest10491-21perf23:30:000
997999180,1cyclictest0-21swapper/000:01:230
997999180,17cyclictest14079-21grep21:55:150
997999180,17cyclictest14079-21grep21:55:140
9987991714,2cyclictest30039-21ssh23:02:233
9987991714,2cyclictest2759-21diskmemload21:19:433
9987991714,2cyclictest21030-21/usr/sbin/munin21:40:143
9987991713,4cyclictest39-21ksoftirqd/321:46:163
998599171,14cyclictest30737-21runrttasks23:41:172
998599170,16cyclictest0-21swapper/222:53:362
9981991715,2cyclictest25-21ksoftirqd/123:06:521
998199170,17cyclictest0-21swapper/121:16:401
997999178,8cyclictest29117-21wc23:40:230
997999174,3cyclictest11692-21apt-get21:15:150
9979991710,6cyclictest20432-21sed20:35:270
997999170,2cyclictest0-21swapper/023:23:390
998799163,2cyclictest0-21swapper/322:13:363
998799162,4cyclictest225-21systemd-journal22:41:543
9987991614,2cyclictest2759-21diskmemload22:21:443
9987991612,3cyclictest381rcuc/320:58:483
998799160,2cyclictest0-21swapper/300:14:403
998599160,16cyclictest0-21swapper/221:44:312
998199160,2cyclictest0-21swapper/122:30:351
997999169,6cyclictest25801-21wc20:50:000
997999169,6cyclictest21795-21grep21:40:180
997999168,7cyclictest5116-21head21:30:160
997999168,7cyclictest29377-21cron00:20:000
997999168,7cyclictest10476-21tr19:10:150
997999168,2cyclictest10503-21chrt21:52:530
997999162,1cyclictest2759-21diskmemload23:35:570
9979991611,1cyclictest0-21swapper/022:51:380
9979991610,5cyclictest26861-21cron19:45:000
9979991610,5cyclictest17240-21cat19:24:590
997999160,6cyclictest0-21swapper/023:55:260
998799159,5cyclictest3780-21dump-pmu-power21:30:003
998799152,9cyclictest4249-21ssh00:05:013
9987991513,1cyclictest11450-21latency_hist23:50:003
9987991513,1cyclictest11450-21latency_hist23:50:003
9987991511,3cyclictest32480-21cut19:55:163
998799151,13cyclictest0-21swapper/323:57:113
9987991510,5cyclictest2759-21diskmemload23:05:543
998199154,5cyclictest225-21systemd-journal00:06:561
9981991511,3cyclictest27359-21ls23:40:011
998199150,13cyclictest0-21swapper/122:47:081
997999158,2cyclictest8286-21ssh00:25:480
997999158,2cyclictest20440-21ssh00:14:310
997999158,2cyclictest18085-21ssh23:15:060
997999157,7cyclictest3988-21perf20:04:590
997999157,7cyclictest31109-21munin-run22:25:010
997999157,2cyclictest16960-21ssh22:55:010
997999155,2cyclictest225-21systemd-journal21:09:590
997999152,2cyclictest225-21systemd-journal19:35:000
997999151,9cyclictest25926-21apt-get20:50:150
997999151,3cyclictest225-21systemd-journal22:35:010
997999150,2cyclictest0-21swapper/023:01:090
997999150,10cyclictest18145-21sh21:38:090
998799149,5cyclictest0-21swapper/321:24:083
998799148,4cyclictest0-21swapper/323:43:243
998799147,6cyclictest2759-21diskmemload00:25:553
9987991412,2cyclictest0-21swapper/323:20:143
9987991411,2cyclictest2759-21diskmemload23:11:363
9985991410,3cyclictest25162-21grep20:45:242
9985991410,3cyclictest22865-21ntpq00:15:212
998599140,2cyclictest0-21swapper/222:13:492
998599140,13cyclictest0-21swapper/223:48:222
998599140,13cyclictest0-21swapper/223:48:212
998199147,6cyclictest0-21swapper/122:43:041
998199145,2cyclictest225-21systemd-journal22:23:051
9981991414,0cyclictest0-21swapper/100:20:311
9981991412,1cyclictest13704-21sh21:55:111
9981991412,1cyclictest13704-21sh21:55:111
998199140,3cyclictest27021-21ps23:00:231
998199140,14cyclictest0-21swapper/123:46:301
998199140,14cyclictest0-21swapper/123:46:291
998199140,13cyclictest0-21swapper/122:50:051
997999148,2cyclictest6641-21ssh23:46:160
997999148,2cyclictest6641-21ssh23:46:160
997999148,2cyclictest14296-21awk19:15:280
997999148,1cyclictest30441-21ssh22:05:100
997999148,1cyclictest27832-21ssh21:24:590
997999147,6cyclictest29996-21cut00:20:150
997999147,2cyclictest7650-21chrt22:10:200
997999147,2cyclictest32476-21taskset19:55:160
997999147,2cyclictest31614-21ssh22:25:130
997999147,2cyclictest24821-21expr22:40:150
997999147,2cyclictest22481-21tr22:00:170
997999147,2cyclictest22481-21tr22:00:170
997999147,2cyclictest12083-21fschecks_time20:20:150
997999146,2cyclictest13177-21cat23:50:190
997999142,3cyclictest225-21systemd-journal19:55:000
997999141,3cyclictest0-21swapper/023:05:160
997999140,9cyclictest11287-21cut23:10:230
997999140,4cyclictest225-21systemd-journal23:35:000
997999140,13cyclictest23454-21gzip19:35:230
998799139,3cyclictest2759-21diskmemload22:30:273
998799138,5cyclictest2759-21diskmemload22:55:293
998799136,6cyclictest23151-21cstates00:35:193
9987991312,1cyclictest2759-21diskmemload23:34:533
9987991312,1cyclictest0-21swapper/323:27:473
9987991311,1cyclictest0-21swapper/320:35:133
9987991310,2cyclictest5142-21cut21:30:173
9987991310,2cyclictest3090-21tr20:00:223
9987991310,1cyclictest2759-21diskmemload22:07:053
998799130,9cyclictest0-21swapper/320:25:143
998599139,3cyclictest30296-21grep00:00:262
998599139,3cyclictest29115-21head21:25:152
998599139,3cyclictest2580-21tail20:00:182
998599139,3cyclictest17595-21grep19:25:162
998599136,5cyclictest20744-21wc21:40:012
998599135,5cyclictest31433-21sed22:05:192
9985991313,0cyclictest0-21swapper/223:54:462
9985991310,2cyclictest9760-21grep20:15:172
9985991310,2cyclictest9385-21sh23:09:592
9985991310,2cyclictest5112-21grep21:30:162
9985991310,2cyclictest30623-21grep00:20:172
9985991310,2cyclictest29573-21grep21:45:142
9985991310,2cyclictest18472-21chrt00:13:002
9985991310,2cyclictest15081-21head00:30:182
9985991310,2cyclictest12389-21uniq21:15:172
9985991310,2cyclictest10845-21idleruntime23:10:192
998599130,3cyclictest0-21swapper/222:59:232
998199139,2cyclictest25021-21cron22:59:591
998199138,4cyclictest684-21ssh23:43:081
9981991310,2cyclictest24137-21tail22:00:281
9981991310,2cyclictest24137-21tail22:00:271
9981991310,2cyclictest10452-21tr19:10:151
998199130,1cyclictest2759-21diskmemload23:33:501
997999138,1cyclictest231272sleep020:43:120
997999137,1cyclictest62522sleep020:09:030
997999136,2cyclictest15394-21grep22:15:140
997999132,6cyclictest8791-21awk20:15:000
998799129,2cyclictest13235-21sh21:55:003
998799129,2cyclictest12312-21cron19:15:003
998799128,3cyclictest22696-21grep19:35:163
998799128,3cyclictest19015-21seq22:36:173
998799126,6cyclictest0-21swapper/321:04:593
998799126,5cyclictest31083-21ssh00:20:203
998799121,4cyclictest7766-21sh00:06:233
9987991210,2cyclictest39-21ksoftirqd/300:30:243
998799120,1cyclictest0-21swapper/319:20:163
998599129,2cyclictest993-21perf23:05:002
998599129,2cyclictest7212-21ls00:25:232
998599129,2cyclictest25506-21sh23:38:442
998599129,2cyclictest24104-21grep22:20:212
998599129,2cyclictest21656-21run-parts22:00:002
998599129,2cyclictest21656-21run-parts21:59:592
998599129,2cyclictest12525-21grep20:20:182
998599129,2cyclictest10531-21date23:30:002
998599129,2cyclictest10123-21taskset22:30:522
998599128,3cyclictest6684-21grep21:50:202
998599128,3cyclictest20897-21tr19:30:202
998599128,2cyclictest16255-21tr22:35:162
998599126,2cyclictest225-21systemd-journal20:54:592
998599122,9cyclictest408-21apache_volume22:45:142
998599120,1cyclictest0-21swapper/200:36:092
998599120,12cyclictest0-21swapper/223:30:532
998199129,2cyclictest9729-21chrt23:29:271
998199129,2cyclictest5593-21seq21:30:201
998199129,2cyclictest3550-21tr21:10:161
998199129,2cyclictest32656-21sed19:55:171
998199129,2cyclictest29542-21grep19:50:151
998199129,2cyclictest16452-21grep20:30:161
998199129,2cyclictest15014-21grep00:30:171
998199129,2cyclictest14352-21idleruntime-cro22:15:011
998199129,2cyclictest14164-21wc21:35:241
998199129,2cyclictest12256-21grep23:50:141
998199129,2cyclictest11800-21sed23:10:271
998199129,2cyclictest10759-21grep20:15:271
998199128,3cyclictest19444-21cut20:35:171
998199128,3cyclictest17955-21sh00:12:351
998199127,4cyclictest28898-21sed21:25:141
998199127,4cyclictest25394-21ssh21:23:051
998199126,3cyclictest13562-21munin-run20:25:011
998199124,6cyclictest2759-21diskmemload22:35:381
998199121,1cyclictest8302-21ssh21:51:101
998199120,11cyclictest27417-21tr19:45:151
997999129,2cyclictest17990-21sort19:25:180
997999129,2cyclictest14047-21grep20:25:150
997999124,2cyclictest10657-21sed20:15:270
997999122,3cyclictest27409-21grep19:45:140
998799119,1cyclictest215642sleep319:31:023
998799118,2cyclictest6013-21chrt20:05:573
998799118,2cyclictest29607-21cut19:50:153
998799118,2cyclictest27237-21grep19:45:153
998799118,2cyclictest12765-21which20:20:203
998799118,2cyclictest11191-21basename20:20:013
998799115,5cyclictest701-21wget21:05:163
998799115,5cyclictest0-21swapper/322:15:143
998799114,6cyclictest55750irq/128-enp2s0-23:53:463
998799114,5cyclictest8769-21sh20:14:593
998799110,1cyclictest0-21swapper/322:46:293
998599119,1cyclictest0-21swapper/221:20:272
998599118,2cyclictest702-21grep21:05:162
998599118,2cyclictest27622-21df19:45:162
998599118,2cyclictest25684-21chrt22:40:192
998599118,2cyclictest22811-21sed20:40:292
998599118,2cyclictest22148-21apache_volume19:35:132
998599118,2cyclictest15751-21sed19:20:192
998599118,2cyclictest1459-21ssh22:25:392
998599118,2cyclictest14236-21sed19:15:282
998599118,2cyclictest10457-21grep19:10:152
998599117,2cyclictest4999-21grep20:05:172
998599110,3cyclictest0-21swapper/223:15:552
998599110,1cyclictest225-21systemd-journal19:40:492
998599110,10cyclictest15480-21ssh22:15:152
998199118,2cyclictest7703-21cat00:25:281
998199118,2cyclictest648-21rt-features22:25:231
998199118,2cyclictest5835-21sed20:05:261
998199118,2cyclictest3001-21ssh00:04:121
998199118,2cyclictest29899-21grep20:55:291
998199118,2cyclictest26149-21ssh00:36:401
998199118,2cyclictest25956-21grep19:40:241
998199118,2cyclictest24463-21sh23:18:441
998199118,2cyclictest24255-21head20:45:151
998199118,2cyclictest23463-21cat20:45:011
998199118,2cyclictest22654-21chrt23:55:431
998199118,2cyclictest22141-21bc19:35:121
998199118,2cyclictest21264-21cut21:40:161
998199117,3cyclictest28020-21sh20:54:591
998199111,9cyclictest38150irq/131-snd_hda22:05:191
997999118,2cyclictest441-21cat21:05:000
997999117,3cyclictest17885-21head20:30:270
998799108,1cyclictest18092-21expr19:25:183
998799107,2cyclictest24477-21uname20:45:173
998799107,2cyclictest14527-21taskset19:17:553
998799102,6cyclictest25410-21grep19:40:193
998799100,9cyclictest16146-21chrt20:30:053
998599108,1cyclictest6749-21egrep20:10:142
998599108,1cyclictest19223-21cat20:35:152
998599108,1cyclictest156602sleep220:26:482
998599107,2cyclictest32557-21sort19:55:172
998599107,2cyclictest3209-21ssh21:10:122
998599107,2cyclictest29721-21ls19:50:162
998599107,2cyclictest29292-21cat20:55:222
998599107,2cyclictest17581-21smart_sda20:30:262
998199108,1cyclictest2315-21cat20:00:161
998199108,1cyclictest14723-21ssh22:15:071
998199107,2cyclictest32350-21grep21:00:281
998199107,2cyclictest18091-21head19:25:181
998199107,2cyclictest13917-21which19:15:241
998199107,2cyclictest1369-21tail21:05:181
998199101,3cyclictest0-21swapper/119:30:141
998199100,9cyclictest16902-21taskset19:22:411
997999108,1cyclictest300982sleep020:56:150
99879991,5cyclictest39-21ksoftirqd/320:40:113
99859997,1cyclictest32006-21grep21:00:242
99819997,1cyclictest15599-21chrt20:25:581
99819996,2cyclictest6811-21cat20:10:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional