You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-04 - 03:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sat Jul 04, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3391210153,43sleep20-21swapper/219:05:522
3391210153,43sleep20-21swapper/219:05:512
361329178,9sleep00-21swapper/019:08:430
361329178,9sleep00-21swapper/019:08:420
338728471,9sleep30-21swapper/319:05:493
338728471,9sleep30-21swapper/319:05:483
341528363,6sleep10-21swapper/119:06:111
341528363,6sleep10-21swapper/119:06:101
277662660,0sleep30-21swapper/323:41:173
380299280,27cyclictest0-21swapper/223:55:462
380299260,23cyclictest0-21swapper/222:15:422
3802992525,0cyclictest0-21swapper/219:54:162
380299250,25cyclictest0-21swapper/221:54:592
380099250,1cyclictest0-21swapper/122:33:361
3800992421,2cyclictest25-21ksoftirqd/119:55:001
380099241,16cyclictest0-21swapper/121:30:251
380299230,20cyclictest0-21swapper/221:28:532
3796992312,6cyclictest2866-21sh00:05:160
380599220,18cyclictest0-21swapper/323:59:543
3802992220,1cyclictest0-21swapper/222:26:482
380299220,22cyclictest0-21swapper/223:30:342
3800992220,1cyclictest25-21ksoftirqd/122:20:441
380099221,12cyclictest141rcu_preempt21:26:181
3802992121,0cyclictest0-21swapper/222:31:592
380099210,21cyclictest0-21swapper/100:34:441
380099210,18cyclictest0-21swapper/122:15:141
380099210,13cyclictest0-21swapper/100:28:281
379699210,21cyclictest0-21swapper/021:40:130
380599200,18cyclictest0-21swapper/319:55:133
380299201,3cyclictest141rcu_preempt22:04:322
380299200,20cyclictest0-21swapper/223:10:402
380299200,19cyclictest0-21swapper/221:20:442
3796992019,1cyclictest0-21swapper/023:10:370
3805991916,2cyclictest39-21ksoftirqd/322:05:443
380599190,19cyclictest0-21swapper/323:51:483
380299191,18cyclictest0-21swapper/222:55:532
380299190,19cyclictest0-21swapper/222:46:472
380299190,18cyclictest0-21swapper/221:36:152
380099198,5cyclictest26410-21cpuspeed_turbos00:00:131
3800991916,2cyclictest38150irq/131-snd_hda19:45:131
3800991913,3cyclictest25-21ksoftirqd/123:37:351
380099191,1cyclictest29171-21diskmemload21:40:561
380099190,2cyclictest0-21swapper/122:26:061
221842190,0sleep049-21kcompactd020:55:020
221842190,0sleep049-21kcompactd020:55:010
380599186,8cyclictest4587-21apt-get22:50:133
380299187,7cyclictest30284-21tr23:05:142
380299180,2cyclictest0-21swapper/223:25:202
380299180,18cyclictest0-21swapper/221:14:222
380299180,17cyclictest0-21swapper/222:44:412
380099187,9cyclictest25-21ksoftirqd/122:58:591
380099185,13cyclictest0-21swapper/121:11:591
380099182,1cyclictest141rcu_preempt22:14:031
3800991813,1cyclictest25-21ksoftirqd/121:36:191
379699184,7cyclictest29182-21perf21:10:000
379699180,0cyclictest0-21swapper/022:43:550
276982180,0sleep00-21swapper/023:41:140
380599172,3cyclictest141rcu_preempt22:28:023
3805991715,1cyclictest39-21ksoftirqd/322:48:553
380599170,14cyclictest0-21swapper/300:08:363
380099173,2cyclictest221irq_work/123:58:221
380099172,1cyclictest141rcu_preempt23:09:291
380099170,0cyclictest141rcu_preempt23:48:461
3805991614,1cyclictest39-21ksoftirqd/321:55:283
3805991612,4cyclictest31094-21kworker/u8:100:03:423
380599160,16cyclictest0-21swapper/323:34:573
3802991615,1cyclictest0-21swapper/221:19:202
380299160,16cyclictest0-21swapper/222:24:522
380299160,15cyclictest0-21swapper/223:17:392
380299160,15cyclictest0-21swapper/221:48:392
380299160,15cyclictest0-21swapper/200:39:562
380299160,14cyclictest0-21swapper/200:30:062
380299160,13cyclictest0-21swapper/222:06:392
3800991614,1cyclictest25-21ksoftirqd/100:15:071
3800991613,2cyclictest25-21ksoftirqd/121:50:001
380099160,1cyclictest0-21swapper/123:50:391
379699161,8cyclictest27761-21apt-get19:59:590
3796991614,2cyclictest0-21swapper/021:21:460
379699160,1cyclictest0-21swapper/023:21:570
380599153,5cyclictest29171-21diskmemload23:25:273
3805991514,1cyclictest0-21swapper/300:17:053
380599151,3cyclictest30544-21cron21:30:003
380599151,10cyclictest0-21swapper/322:02:143
3805991510,3cyclictest29171-21diskmemload00:27:353
380599150,5cyclictest141rcu_preempt21:54:263
380299151,1cyclictest29171-21diskmemload23:51:002
380299150,15cyclictest0-21swapper/223:45:242
380299150,15cyclictest0-21swapper/223:41:482
380299150,15cyclictest0-21swapper/221:44:092
380299150,15cyclictest0-21swapper/200:17:382
380099159,6cyclictest38150irq/131-snd_hda00:21:011
3800991514,1cyclictest25-21ksoftirqd/123:25:311
3800991514,1cyclictest0-21swapper/121:55:391
3800991512,3cyclictest0-21swapper/120:05:001
3800991511,2cyclictest25-21ksoftirqd/100:12:121
380099150,15cyclictest0-21swapper/122:08:431
379699155,5cyclictest15501-21perf23:35:000
3796991511,3cyclictest0-21swapper/023:05:090
380599143,10cyclictest14158-21python20:35:243
3805991412,1cyclictest391ksoftirqd/322:19:333
380599140,3cyclictest141rcu_preempt20:25:153
380599140,2cyclictest141rcu_preempt00:30:473
380599140,1cyclictest0-21swapper/322:30:113
380599140,14cyclictest0-21swapper/321:34:403
380299149,3cyclictest28756-21head21:05:302
3802991414,0cyclictest0-21swapper/222:12:432
3802991413,1cyclictest0-21swapper/222:52:542
3802991413,1cyclictest0-21swapper/200:25:432
3800991411,2cyclictest32636-21sh23:45:001
380099140,14cyclictest0-21swapper/119:25:141
379699147,7cyclictest0-21swapper/022:13:150
379699146,7cyclictest15098-21munin-run21:40:000
379699146,7cyclictest13753-21sh23:52:230
3796991410,2cyclictest26512-21wc22:25:000
379699140,1cyclictest0-21swapper/021:50:090
379699140,13cyclictest0-21swapper/022:51:050
380599138,2cyclictest4627-21taskset21:14:323
380599136,1cyclictest0-21swapper/323:20:403
380599134,8cyclictest1513-21perl23:45:163
380599134,3cyclictest20134-21apt-get22:40:153
380299139,3cyclictest26880-21grep19:55:202
380299139,2cyclictest22659-21tail23:00:182
3802991310,2cyclictest8536-21perf19:20:002
3802991310,2cyclictest6741-21grep20:20:182
3802991310,2cyclictest16516-21ssh22:00:002
3802991310,2cyclictest1328-21ssh00:23:262
3802991310,2cyclictest11408-21tr19:25:142
380099136,3cyclictest0-21swapper/122:40:151
380099136,1cyclictest16992-21apt-get22:00:131
380099133,4cyclictest0-21swapper/119:10:141
380099131,3cyclictest141rcu_preempt23:10:321
3800991312,1cyclictest38150irq/131-snd_hda22:53:041
380099130,3cyclictest141rcu_preempt23:30:201
379699139,3cyclictest6627-21cat20:20:170
379699139,3cyclictest18267-21readlink19:40:010
379699139,3cyclictest13984-21grep19:30:120
3796991312,1cyclictest0-21swapper/000:26:360
3796991311,2cyclictest0-21swapper/022:57:240
3796991310,2cyclictest5149-21latency_hist20:20:000
3796991310,2cyclictest20615-21grep20:50:180
380599129,2cyclictest24950-21expr21:00:143
380599129,2cyclictest21626-21ssh23:19:223
380599129,2cyclictest21379-21ssh23:00:093
380599129,2cyclictest16204-21ssh00:12:563
380599129,2cyclictest15345-21egrep20:40:143
380599129,2cyclictest14537-21grep21:20:153
380599129,2cyclictest13109-21taskset19:27:523
380599128,1cyclictest141rcu_preempt00:36:353
380599127,4cyclictest39-21ksoftirqd/319:40:013
380599125,6cyclictest15604-21apache_volume21:40:113
380599124,5cyclictest29171-21diskmemload23:13:283
380599121,10cyclictest0-21swapper/322:10:403
3805991210,1cyclictest1270-21grep20:10:163
380599120,9cyclictest0-21swapper/321:39:273
380599120,4cyclictest0-21swapper/300:20:303
380599120,11cyclictest7171-21head21:15:213
380599120,11cyclictest4476-21cat20:15:243
380599120,11cyclictest13512-21grep22:35:263
380299129,2cyclictest9862-21wc19:20:192
380299129,2cyclictest5731-21grep19:10:272
380299129,2cyclictest32147-21wc20:05:242
380299129,2cyclictest26404-21tr00:00:132
380299129,2cyclictest25900-21grep21:00:222
380299129,2cyclictest21656-21grep19:45:172
380299129,2cyclictest20250-21sed19:40:262
380299129,2cyclictest13477-21sort22:35:262
380299129,2cyclictest11758-21grep00:10:172
380299128,3cyclictest3356-21cstates20:15:142
380299128,2cyclictest31173-21ssh21:30:122
380299127,4cyclictest17414-21grep23:35:192
380299125,3cyclictest225-21systemd-journal19:33:212
380299122,3cyclictest0-21swapper/200:09:192
380099129,2cyclictest9639-21sed20:25:261
380099129,2cyclictest25667-21sh23:20:571
380099129,2cyclictest1976-21wc20:10:221
380099129,2cyclictest11744-21cut22:35:131
380099128,1cyclictest21582-21cut00:35:151
3800991211,1cyclictest0-21swapper/123:03:201
3800991210,1cyclictest22455-21cron23:20:001
380099120,4cyclictest225-21systemd-journal21:10:001
380099120,1cyclictest0-21swapper/121:15:171
380099120,11cyclictest5143-21apt-get20:20:001
380099120,10cyclictest0-21swapper/121:52:151
379699129,2cyclictest8831-21basename19:20:120
379699129,2cyclictest872-21grep23:25:240
379699129,2cyclictest8614-21grep20:25:170
379699129,2cyclictest3394-21ssh00:25:010
379699129,2cyclictest28488-21tr20:00:150
379699129,2cyclictest27376-21basename00:20:000
379699129,2cyclictest22769-21tr21:25:140
379699129,2cyclictest20772-21ssh22:01:490
379699129,2cyclictest17763-21grep20:45:140
379699129,2cyclictest16378-21grep20:40:220
379699129,2cyclictest15782-21cut23:15:210
379699129,2cyclictest14922-21wc20:40:000
379699129,2cyclictest10647-21tr20:30:140
379699128,3cyclictest6072-21tr21:15:140
379699122,7cyclictest23754-21ssh23:58:350
379699120,1cyclictest0-21swapper/023:45:000
379699120,12cyclictest0-21swapper/000:30:300
379699120,11cyclictest11138-21cron22:35:010
380599119,2cyclictest8383-21chrt19:18:223
380599119,2cyclictest27835-21chrt21:05:183
380599119,2cyclictest18621-21sort22:20:143
380599119,2cyclictest16551-21cat23:35:143
380599119,1cyclictest29873-21sh23:05:093
380599119,1cyclictest174912sleep320:45:063
380599118,2cyclictest30940-21cat20:05:133
380599118,2cyclictest25454-21grep21:45:213
380599118,2cyclictest23925-21uniq19:50:163
380599118,2cyclictest21674-21head19:45:173
380599118,2cyclictest16213-21ssh22:56:203
380599116,2cyclictest225-21systemd-journal19:25:013
380299119,1cyclictest133412sleep220:35:152
380299118,2cyclictest28361-21wget20:00:142
380299118,2cyclictest19636-21apt20:50:002
380299118,2cyclictest16163-21cut19:35:132
380299110,10cyclictest24538-21awk23:20:212
380099119,2cyclictest25997-21fgrep21:00:221
380099119,2cyclictest21373-21cat20:50:271
380099119,2cyclictest19059-21sh21:22:381
380099119,1cyclictest9915-21cut19:20:191
380099119,1cyclictest24378-21cron21:00:011
380099119,1cyclictest24378-21cron21:00:001
380099119,1cyclictest184962sleep119:40:091
380099119,1cyclictest18243-21grep20:45:161
380099119,1cyclictest16295-21grep20:40:211
380099119,1cyclictest14988-21tr19:30:191
380099118,2cyclictest6597-21wc20:20:171
380099118,2cyclictest28282-21ssh22:45:031
380099118,2cyclictest25788-21cut19:55:121
380099112,6cyclictest2206-21ssh00:05:121
380099112,2cyclictest225-21systemd-journal19:36:281
379699118,2cyclictest32561-21ssh00:03:460
379699118,2cyclictest30991-21ssh22:45:470
379699118,2cyclictest30209-21expr21:10:170
379699118,2cyclictest2800-21ssh21:32:020
379699118,2cyclictest27841-21grep22:25:160
379699118,2cyclictest25821-21cat21:00:210
379699118,2cyclictest25363-21sh23:01:520
379699118,2cyclictest24763-21grep21:45:150
379699118,2cyclictest24376-21grep19:50:190
379699118,2cyclictest18512-21ssh22:39:100
379699118,2cyclictest17729-21rm23:35:210
379699118,2cyclictest13404-21ssh22:16:350
379699118,2cyclictest13290-21ssh21:57:330
379699115,5cyclictest20830-21apt-get00:35:120
379699110,1cyclictest0-21swapper/022:07:290
380599109,1cyclictest5515-21sed20:20:013
380599109,1cyclictest28731-21expr20:00:153
380599108,1cyclictest4373-21cat19:10:153
380599108,1cyclictest18492-21taskset19:40:053
380599107,2cyclictest14179-21expr19:30:143
380599102,5cyclictest0-21swapper/320:55:123
380599102,5cyclictest0-21swapper/320:55:113
380599100,6cyclictest21948-21sshd20:54:223
380599100,10cyclictest10686-21expr20:30:133
380299108,1cyclictest21211-21grep20:50:232
380299107,2cyclictest2600-21taskset20:13:062
380299107,2cyclictest0-21swapper/220:40:212
380299103,5cyclictest10525-21grep20:30:132
380099109,1cyclictest11895-21wc20:30:251
380099108,2cyclictest32224-21sed20:05:241
380099108,1cyclictest8515-21cron19:20:001
380099107,2cyclictest14466-21head20:35:251
379699108,2cyclictest8123-21awk19:15:260
379699108,1cyclictest31604-21cat20:05:180
379699108,1cyclictest22796-21taskset19:47:200
379699108,1cyclictest18834-21cat19:40:150
379699108,1cyclictest11602-21cat00:10:170
379699107,2cyclictest12263-21uname19:25:190
38029996,2cyclictest8735-21grep20:25:172
38029990,5cyclictest24166-21sshd20:57:222
38029990,5cyclictest24166-21sshd20:57:212
37969997,2cyclictest2483-21chrt20:11:320
37969997,1cyclictest4029-21taskset19:10:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional