You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-05 - 11:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Sun Apr 05, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31339210087,9sleep00-21swapper/019:06:100
3136628561,8sleep20-21swapper/219:06:322
3133028564,6sleep10-21swapper/119:06:031
3133228167,8sleep30-21swapper/319:06:053
181662530,0sleep00-21swapper/022:04:040
3174699300,29cyclictest0-21swapper/222:52:582
3174699300,29cyclictest0-21swapper/222:03:022
3174699290,28cyclictest0-21swapper/221:21:422
3174699280,1cyclictest0-21swapper/221:44:082
3174699260,22cyclictest0-21swapper/221:34:522
3174699250,24cyclictest0-21swapper/222:35:052
31744992422,2cyclictest94-21kswapd021:39:221
3174499240,4cyclictest0-21swapper/121:22:221
3174099241,4cyclictest141rcu_preempt23:05:130
3174699230,22cyclictest0-21swapper/222:28:082
31744992321,1cyclictest0-21swapper/123:50:501
3174499230,23cyclictest0-21swapper/121:51:261
3174699220,22cyclictest0-21swapper/223:29:482
3174699220,20cyclictest30149-21sh00:07:252
3174499220,2cyclictest0-21swapper/122:48:271
3174499220,19cyclictest0-21swapper/123:49:381
267792220,0sleep30-21swapper/300:25:103
3174699210,21cyclictest0-21swapper/200:32:592
31740992121,0cyclictest0-21swapper/023:04:570
3174699200,19cyclictest0-21swapper/200:28:312
31750991912,4cyclictest39-21ksoftirqd/322:13:193
31746991918,1cyclictest0-21swapper/222:57:112
3174699190,1cyclictest0-21swapper/221:36:392
3174699190,19cyclictest0-21swapper/223:13:302
3174699190,0cyclictest0-21swapper/200:39:302
3174699190,0cyclictest0-21swapper/200:21:422
3174699190,0cyclictest0-21swapper/200:21:422
3174499190,19cyclictest0-21swapper/121:12:351
31750991812,5cyclictest24945-21diskmemload23:02:023
3175099181,1cyclictest141rcu_preempt23:07:423
3174699180,2cyclictest0-21swapper/200:18:492
3174099184,7cyclictest17565-21sh20:55:000
3174099181,10cyclictest0-21swapper/022:45:140
3174699170,17cyclictest0-21swapper/223:01:392
3174499174,9cyclictest11852-21systemctl22:20:001
3174499171,11cyclictest0-21swapper/122:20:121
3174499170,6cyclictest0-21swapper/123:15:121
3174499170,1cyclictest0-21swapper/100:09:501
3174099170,17cyclictest0-21swapper/000:22:140
3174099170,17cyclictest0-21swapper/000:22:140
3174699160,16cyclictest0-21swapper/200:02:292
3174499163,11cyclictest28757-21taskset21:30:391
3174499160,15cyclictest0-21swapper/121:40:211
3174099164,6cyclictest12779-21grep22:40:000
31740991615,1cyclictest0-21swapper/021:41:580
3174099160,12cyclictest0-21swapper/021:05:000
31750991514,1cyclictest0-21swapper/321:43:313
31750991513,2cyclictest28877-21kworker/3:121:10:323
3175099150,14cyclictest0-21swapper/323:30:143
3174699152,11cyclictest28129-21apt-get22:10:122
3174699150,15cyclictest0-21swapper/221:48:362
3174699150,15cyclictest0-21swapper/221:17:262
3174699150,14cyclictest0-21swapper/223:07:362
3174699150,14cyclictest0-21swapper/221:58:432
31744991511,2cyclictest4587-21wc00:30:281
31740991510,3cyclictest27125-21head20:05:180
3175099144,3cyclictest0-21swapper/319:20:123
31750991410,1cyclictest0-21swapper/322:45:473
3175099140,13cyclictest12120-21grep00:15:273
31746991413,1cyclictest0-21swapper/221:29:292
31746991411,2cyclictest13446-21ssh22:20:182
31746991410,3cyclictest21159-21cron19:55:002
31746991410,2cyclictest21568-21cut19:55:152
3174699140,14cyclictest0-21swapper/223:31:352
3174699140,14cyclictest0-21swapper/223:18:402
3174699140,12cyclictest22860-21tr21:05:142
3174699140,12cyclictest14828-21head19:40:162
3174699140,12cyclictest0-21swapper/223:46:302
3174499147,3cyclictest27215-21ssh00:25:141
31744991411,2cyclictest31833-21wc22:50:261
3174499140,13cyclictest0-21swapper/122:30:591
3174499140,12cyclictest0-21swapper/122:40:181
3174499140,12cyclictest0-21swapper/122:10:121
3174499140,11cyclictest0-21swapper/119:49:551
3174099149,3cyclictest14198-21grep20:45:200
3174099149,3cyclictest1184-21sh00:10:000
3174099141,7cyclictest2808-21perf21:55:000
31740991410,3cyclictest18836-21cat19:50:000
31740991410,2cyclictest3685-21grep20:25:150
3174099140,8cyclictest0-21swapper/023:30:180
3174099140,2cyclictest0-21swapper/022:25:120
3174099140,0cyclictest0-21swapper/023:20:380
3175099136,4cyclictest39-21ksoftirqd/321:45:203
3175099136,4cyclictest0-21swapper/300:23:043
3175099136,4cyclictest0-21swapper/300:23:043
3175099132,11cyclictest1097-31find00:10:383
31750991311,2cyclictest39-21ksoftirqd/322:20:323
31750991310,2cyclictest4100-21perf19:20:003
31750991310,2cyclictest2772-21cron21:55:003
31750991310,2cyclictest19215-21grep00:00:263
3175099130,12cyclictest0-21swapper/321:20:133
3174699139,2cyclictest4309-21head20:25:182
31746991310,2cyclictest610-21grep23:50:152
31746991310,2cyclictest16183-21grep20:50:172
31746991310,2cyclictest11935-21sed20:40:202
3174699130,13cyclictest0-21swapper/223:20:472
3174499139,2cyclictest11391-21wc00:15:201
3174499138,2cyclictest225-21systemd-journal23:40:001
31744991312,1cyclictest0-21swapper/100:05:001
31744991310,2cyclictest6627-21wc22:35:251
31744991310,2cyclictest38150irq/131-snd_hda23:35:241
31744991310,2cyclictest3390-21ssh00:10:241
31744991310,2cyclictest15418-21wc19:40:221
3174499130,2cyclictest24945-21diskmemload23:27:371
3174499130,12cyclictest18912-21tr00:20:171
3174499130,12cyclictest18912-21tr00:20:171
3174499130,12cyclictest0-21swapper/123:03:111
3174499130,0cyclictest0-21swapper/121:49:051
3174099139,3cyclictest7223-21tr23:15:130
3174099139,3cyclictest2426-21grep20:20:220
3174099139,2cyclictest861-21turbostat20:20:000
3174099133,5cyclictest1040-21munin-run21:15:000
3174099131,6cyclictest0-21swapper/019:25:120
31740991310,2cyclictest4040-21ssh22:15:110
31740991310,2cyclictest25494-21grep20:00:270
31740991310,2cyclictest2104-21basename19:15:140
31740991310,2cyclictest13899-21cat19:40:010
3174099130,1cyclictest141rcu_preempt00:17:140
3174099130,12cyclictest30396-21grep20:10:260
25952130,0sleep00-21swapper/000:30:070
3175099129,2cyclictest8503-21df23:35:143
3175099129,2cyclictest8290-21ls23:15:203
3175099129,2cyclictest6630-21cut20:30:173
3175099129,2cyclictest3938-21cat00:30:213
3175099129,2cyclictest30923-21head22:50:173
3175099129,2cyclictest30781-21cat22:30:253
3175099129,2cyclictest25689-21wc23:45:203
3175099129,2cyclictest20964-21cat22:05:203
3175099129,2cyclictest19326-21grep19:50:133
3175099129,2cyclictest18412-21egrep21:25:143
3175099129,2cyclictest16196-21cron19:45:003
3175099129,2cyclictest16065-21cut20:50:173
3175099129,2cyclictest13865-21sleep23:00:003
3175099129,2cyclictest12872-21ssh00:35:243
3175099129,2cyclictest10217-21cron21:40:003
3175099128,3cyclictest1198-21sleep00:10:003
3175099127,4cyclictest9947-21sh21:59:083
3175099124,4cyclictest225-21systemd-journal19:29:593
3175099120,11cyclictest1970-21cut20:20:163
3175099120,11cyclictest15812-21df23:20:153
3174699129,2cyclictest8444-21basename20:35:142
3174699129,2cyclictest7793-21chrt22:16:492
3174699129,2cyclictest27142-21tail20:05:182
3174699129,2cyclictest2600-21sort19:15:162
3174699129,2cyclictest19520-21grep20:55:262
3174699128,3cyclictest9102-21sh23:35:202
3174699128,3cyclictest30886-21cron20:15:002
3174699125,4cyclictest30870-21sed22:30:262
3174699124,8cyclictest32-21ksoftirqd/223:43:242
3174699121,9cyclictest23243-21sensors22:45:242
3174499129,2cyclictest8944-21grep20:35:161
3174499129,2cyclictest8456-21grep19:25:251
3174499129,2cyclictest4596-21tr19:20:131
3174499129,2cyclictest32023-21grep19:10:121
3174499129,2cyclictest31297-21grep20:15:131
3174499129,2cyclictest30010-21tr20:10:211
3174499129,2cyclictest24574-21cut20:00:171
3174499129,2cyclictest19492-21grep20:55:261
3174499129,2cyclictest13467-21grep22:00:261
3174499129,2cyclictest13082-21grep00:35:251
3174499129,2cyclictest11447-21basename19:34:591
3174499128,3cyclictest1639-21grep20:20:161
31744991211,1cyclictest0-21swapper/123:20:111
31744991210,1cyclictest24945-21diskmemload23:34:201
3174499120,1cyclictest0-21swapper/123:11:171
3174499120,1cyclictest0-21swapper/121:59:451
3174099129,2cyclictest6467-21cron19:25:000
3174099129,2cyclictest3461-21diskmemload23:12:390
3174099129,2cyclictest31481-21grep22:50:210
3174099129,2cyclictest24399-21tail23:25:160
3174099129,2cyclictest2064-21wc00:10:140
3174099129,2cyclictest19015-21ssh21:25:170
3174099129,2cyclictest16193-21cron19:45:000
3174099129,2cyclictest14822-21ssh22:40:210
3174099128,3cyclictest9939-21grep20:35:250
3174099124,5cyclictest586-21sshd22:12:430
3174099124,4cyclictest225-21systemd-journal22:00:000
3174099121,6cyclictest16800-21awk00:00:000
31740991210,1cyclictest27389-21grep21:30:180
3174099120,12cyclictest0-21swapper/023:41:100
3175099119,1cyclictest5813-21head22:35:183
3175099119,1cyclictest22145-21head19:55:163
3175099118,2cyclictest29048-21cut20:10:143
3175099118,2cyclictest27599-21grep21:30:203
3175099118,2cyclictest24441-21sed21:05:263
3175099118,2cyclictest23943-21ssh22:26:263
3175099118,2cyclictest23497-21sh23:25:103
3175099118,2cyclictest22407-21ssh23:43:413
3175099118,2cyclictest18293-21expr20:55:153
3175099118,2cyclictest16815-21ssh22:41:393
3175099118,2cyclictest15265-21ssh22:01:463
3175099118,2cyclictest1416-21taskset19:12:163
3175099118,2cyclictest13820-21tail20:45:173
3175099116,4cyclictest4758-21fschecks_count22:15:163
3175099111,9cyclictest21024-21diskstats21:00:173
3175099110,10cyclictest948-21chrt23:10:443
3174699118,2cyclictest6155-21cat20:30:162
3174699118,2cyclictest4026-21ssh00:10:412
3174699118,2cyclictest32426-21wc20:15:212
3174699118,2cyclictest29655-21ssh21:50:402
3174699118,2cyclictest28184-21ssh21:10:522
3174699118,2cyclictest19892-21ssh22:05:132
3174699118,2cyclictest17245-21ssh22:41:592
3174699118,2cyclictest13194-21cut20:45:142
3174699118,2cyclictest10044-21grep19:30:172
3174699118,2cyclictest0-21swapper/219:10:252
31746991110,1cyclictest0-21swapper/219:45:132
3174699110,5cyclictest0-21swapper/223:55:142
3174699110,10cyclictest21535-21cut21:00:212
3174499118,2cyclictest6112-21cut22:55:131
3174499118,2cyclictest4313-21grep20:25:181
3174499118,2cyclictest28535-21sleep20:10:001
3174499118,2cyclictest2393-21head21:15:171
3174499118,2cyclictest22544-21ssh23:05:071
3174499118,2cyclictest19290-21grep19:50:131
3174499118,2cyclictest18307-21hostname21:25:141
3174499118,2cyclictest13702-21sh23:57:361
3174499118,2cyclictest12739-21seq20:44:021
3174499117,2cyclictest23561-21tail21:05:181
3174499111,2cyclictest0-21swapper/122:25:131
3174099118,2cyclictest6167-21ssh23:53:140
3174099118,2cyclictest31390-21ssh22:30:460
3174099118,2cyclictest2722-21grep21:35:150
3174099118,2cyclictest26548-21tail23:45:260
3174099118,2cyclictest2417-21head21:15:170
3174099118,2cyclictest23049-21expr21:05:150
3174099118,2cyclictest19844-21grep21:45:160
3174099118,2cyclictest19308-21grep19:50:130
3174099118,2cyclictest18563-21grep20:55:180
3174099118,2cyclictest18293-21ssh00:00:190
3174099118,2cyclictest17618-21ssh22:22:590
3174099118,2cyclictest11548-21cat00:35:140
3174099118,2cyclictest10547-21ssh21:20:150
3174099117,3cyclictest21297-21wc22:05:220
3174099110,6cyclictest0-21swapper/023:35:480
3175099108,2cyclictest13666-21chrt19:38:183
3175099108,1cyclictest5812sleep320:16:453
3175099108,1cyclictest24401-21sh20:00:163
3175099108,1cyclictest1306-21ssh21:15:043
3175099107,3cyclictest39-21ksoftirqd/320:35:123
3175099107,2cyclictest9733-21cat19:30:153
3175099107,2cyclictest8950-21sh23:55:123
3175099107,2cyclictest4354-21head20:25:183
3175099107,2cyclictest345-21rm23:50:083
3175099107,2cyclictest11344-21sort20:40:153
3174699108,1cyclictest5728-21run-parts19:20:222
3174699107,2cyclictest7677-21uname19:25:172
3174699107,2cyclictest1282-21wget20:20:142
3174699106,2cyclictest25948-21ls20:05:002
3174699100,9cyclictest13538-21taskset19:36:402
3174499108,1cyclictest7053-21sed20:30:201
3174499108,1cyclictest2484-21sed19:15:151
3174499108,1cyclictest141072sleep120:45:201
3174499107,2cyclictest22633-21ssh22:06:031
3174499107,2cyclictest21451-21sed21:00:201
3174499107,2cyclictest17069-21grep20:50:261
3174499106,3cyclictest23532-21cron20:00:001
3174099108,1cyclictest5951-21apache_accesses20:30:120
3174099108,1cyclictest32000-21cut19:10:120
3174099108,1cyclictest26744-21ssh00:25:070
3174099108,1cyclictest22063-21cut19:55:160
3174099108,1cyclictest10655-21expr19:30:230
3174099107,2cyclictest7540-21sendmail_mailst22:55:210
3174099107,2cyclictest12235-21cat20:40:240
317509997,1cyclictest26471-21aten2_r7power_e20:05:163
317509997,1cyclictest16613-21egrep19:45:133
317449997,1cyclictest11705-21apt19:35:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional