You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-07 - 01:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Sat Jun 06, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1893629654,37sleep20-21swapper/219:09:302
1881629369,8sleep00-21swapper/019:07:540
1684428575,6sleep10-21swapper/119:05:061
1894428469,9sleep30-21swapper/319:09:353
19071992524,1cyclictest39-21ksoftirqd/321:27:393
19071992323,0cyclictest0-21swapper/300:27:073
19071992321,1cyclictest39-21ksoftirqd/323:07:093
19071992320,2cyclictest39-21ksoftirqd/322:35:423
19071992318,5cyclictest39-21ksoftirqd/323:39:593
19065992320,2cyclictest23616-21perl00:10:151
1906199230,22cyclictest0-21swapper/023:02:340
1906199230,22cyclictest0-21swapper/022:28:400
19071992216,2cyclictest39-21ksoftirqd/321:35:273
19065992219,2cyclictest13973-21irqstats23:45:181
1906599220,1cyclictest0-21swapper/120:30:121
1906199220,0cyclictest0-21swapper/022:39:360
1906599211,1cyclictest0-21swapper/120:00:141
1906199210,21cyclictest0-21swapper/022:05:270
19071992018,2cyclictest39-21ksoftirqd/322:06:473
1907199201,4cyclictest39-21ksoftirqd/320:45:213
19065992019,1cyclictest0-21swapper/121:00:131
19061992017,2cyclictest29887-21hddtemp_smartct21:20:190
1906199200,20cyclictest0-21swapper/021:33:540
1906199200,20cyclictest0-21swapper/021:33:530
1907199199,5cyclictest39-21ksoftirqd/323:49:363
19071991919,0cyclictest0-21swapper/322:25:533
19071991918,1cyclictest16636-21kworker/u8:121:57:023
19071991917,2cyclictest39-21ksoftirqd/321:17:193
1906599190,1cyclictest0-21swapper/100:35:131
1906199192,15cyclictest1125-21apache_accesses22:40:110
19061991918,1cyclictest0-21swapper/021:12:380
1906199190,16cyclictest0-21swapper/000:01:590
1907199184,9cyclictest39-21ksoftirqd/321:42:093
19071991814,1cyclictest39-21ksoftirqd/300:13:183
1907199180,1cyclictest0-21swapper/323:26:513
19067991817,1cyclictest0-21swapper/221:44:132
1906799180,1cyclictest0-21swapper/222:21:032
1906599189,6cyclictest141rcu_preempt23:32:481
19065991816,1cyclictest0-21swapper/120:05:121
1906599180,18cyclictest0-21swapper/100:30:081
1906199186,1cyclictest11658-21diskmemload22:12:550
1906199185,6cyclictest21428-21perf19:15:010
1907199174,1cyclictest141rcu_preempt21:34:163
1907199174,1cyclictest141rcu_preempt21:34:153
1907199173,4cyclictest39-21ksoftirqd/322:50:543
19071991716,1cyclictest0-21swapper/322:19:043
19071991713,2cyclictest381rcuc/319:15:143
1907199171,1cyclictest0-21swapper/323:15:523
1907199171,14cyclictest39-21ksoftirqd/322:45:133
1906799171,5cyclictest141rcu_preempt23:00:102
1906799170,16cyclictest0-21swapper/222:44:012
1906599171,1cyclictest10042-21df22:45:141
1906199178,8cyclictest16759-21ssh22:30:140
1907199163,10cyclictest39-21ksoftirqd/322:04:233
1907199161,3cyclictest39-21ksoftirqd/322:44:423
1907199160,4cyclictest39-21ksoftirqd/323:13:483
1907199160,3cyclictest0-21swapper/322:31:263
19067991613,2cyclictest5569-21irqstats23:40:182
19065991616,0cyclictest0-21swapper/122:31:471
19065991612,3cyclictest0-21swapper/123:22:021
1906599160,1cyclictest0-21swapper/122:22:541
1906199160,1cyclictest0-21swapper/021:19:380
1907199157,3cyclictest39-21ksoftirqd/323:20:313
1907199156,6cyclictest39-21ksoftirqd/300:38:253
19071991510,2cyclictest39-21ksoftirqd/322:58:143
1907199150,15cyclictest0-21swapper/321:47:453
1906799150,1cyclictest0-21swapper/221:12:392
1906599157,2cyclictest537-21snmpd23:54:231
1906599153,8cyclictest11658-21diskmemload21:15:131
1906599151,1cyclictest0-21swapper/122:08:131
1906599151,13cyclictest0-21swapper/122:10:541
1907199147,6cyclictest7220-21ssh00:20:093
19071991411,2cyclictest3998-21grep20:50:253
19071991411,2cyclictest17277-21wc20:10:253
19071991411,2cyclictest15289-21cut21:50:163
19071991411,0cyclictest39-21ksoftirqd/323:52:423
19071991410,2cyclictest7782-21cut21:00:153
19071991410,1cyclictest0-21swapper/322:14:153
1907199140,9cyclictest15328-21sh21:11:203
1906799148,2cyclictest225-21systemd-journal00:18:532
1906799143,6cyclictest23083-21grep00:10:132
19067991413,1cyclictest0-21swapper/223:17:352
19067991411,2cyclictest31904-21cron22:19:592
19067991410,3cyclictest24427-21wc21:55:222
1906799140,13cyclictest0-21swapper/219:10:112
19065991413,1cyclictest11658-21diskmemload21:56:381
19065991413,1cyclictest0-21swapper/123:28:181
1906599140,9cyclictest0-21swapper/121:32:411
1906599140,9cyclictest0-21swapper/121:32:401
1906199149,3cyclictest24056-21ls19:20:000
1906199144,5cyclictest225-21systemd-journal00:14:590
19061991414,0cyclictest0-21swapper/023:09:430
1906199140,14cyclictest0-21swapper/021:55:120
1906199140,0cyclictest0-21swapper/023:27:050
1907199139,3cyclictest6958-21ssh00:00:193
1907199138,5cyclictest39-21ksoftirqd/300:33:233
1907199137,6cyclictest13960-21kworker/u8:100:06:193
1907199134,1cyclictest141rcu_preempt23:03:553
19071991312,1cyclictest0-21swapper/323:40:083
19071991310,2cyclictest860-21sed00:15:273
19071991310,2cyclictest7833-21grep19:50:253
19071991310,2cyclictest31387-21wc20:40:213
19071991310,2cyclictest25024-21cron20:29:593
19071991310,2cyclictest24362-21tr19:20:133
19071991310,2cyclictest22592-21sh20:24:593
1906799139,3cyclictest24018-21grep20:25:182
1906799131,5cyclictest0-21swapper/221:15:132
19067991312,1cyclictest0-21swapper/222:30:542
19067991312,1cyclictest0-21swapper/222:07:362
19067991312,1cyclictest0-21swapper/221:49:342
19067991312,1cyclictest0-21swapper/200:27:302
19067991310,2cyclictest31186-21tr20:40:192
19067991310,2cyclictest27395-21cron20:35:002
19067991310,2cyclictest23192-21grep23:50:262
19067991310,2cyclictest19701-21grep20:15:252
19067991310,2cyclictest12941-21cat21:30:012
19067991310,2cyclictest12529-21diskstats23:25:152
1906599139,3cyclictest31981-21head19:35:161
1906599139,3cyclictest31981-21head19:35:151
1906599139,3cyclictest31641-21grep22:00:131
1906599139,3cyclictest19189-21cat22:50:181
1906599132,10cyclictest7530-21ssh21:25:551
19065991310,2cyclictest31351-21cat21:40:171
19065991310,2cyclictest30520-21sed19:30:261
19065991310,2cyclictest2916-21df23:00:131
19065991310,2cyclictest2181-21head19:40:141
1906599130,1cyclictest0-21swapper/100:20:591
1906599130,1cyclictest0-21swapper/100:18:561
1906199139,3cyclictest30002-21tr19:30:200
1906199131,6cyclictest11669-21perf21:10:000
19061991313,0cyclictest0-21swapper/000:06:310
1906199131,1cyclictest0-21swapper/000:19:360
19061991310,2cyclictest27343-21grep22:55:170
19061991310,2cyclictest24930-21grep22:15:170
19061991310,2cyclictest21030-21ssh23:30:170
19061991310,2cyclictest1815-21sort22:20:260
19061991310,2cyclictest16949-21ntpq00:25:200
1906199130,1cyclictest0-21swapper/022:49:300
1906199130,13cyclictest0-21swapper/023:58:550
1906199130,11cyclictest30838-21grep20:40:160
1907199129,2cyclictest30250-21cat21:20:213
1907199129,2cyclictest27957-21grep20:35:143
1907199129,2cyclictest20528-21tr19:10:213
1907199129,2cyclictest11186-21tr20:00:153
1907199129,2cyclictest10257-21head19:55:263
1907199128,3cyclictest27550-21cat20:35:003
1907199125,7cyclictest0-21swapper/319:40:113
1907199124,7cyclictest39-21ksoftirqd/322:20:443
1906799129,2cyclictest8382-21head21:00:202
1906799129,2cyclictest7421-21cut00:20:122
1906799129,2cyclictest6878-21munin-plugin-st21:00:002
1906799129,2cyclictest31096-21cat19:35:012
1906799129,2cyclictest28172-21sed19:25:252
1906799129,2cyclictest25454-21grep22:35:152
1906799129,2cyclictest21689-21tr23:30:222
1906799129,2cyclictest15192-21grep21:30:262
1906799129,2cyclictest15192-21grep21:30:262
1906799129,2cyclictest15044-21ssh23:45:302
1906799129,2cyclictest12024-21grep23:05:192
1906799127,2cyclictest5685-21apache_volume00:00:112
19067991210,1cyclictest31339-21ssh22:00:122
1906799120,12cyclictest0-21swapper/223:11:172
1906799120,11cyclictest27940-21tr20:35:142
1906599129,2cyclictest30060-21grep23:55:151
1906599129,2cyclictest25817-21grep19:20:261
1906599129,2cyclictest2462-21tr20:50:131
1906599129,2cyclictest19910-21grep23:10:151
1906599129,2cyclictest14900-21tr21:50:141
1906599129,2cyclictest14025-21grep21:10:261
1906599129,2cyclictest10278-21grep22:25:251
1906599123,7cyclictest28941-21/usr/sbin/munin21:20:181
1906599120,11cyclictest0-21swapper/123:38:081
1906199129,2cyclictest9344-21grep19:55:160
1906199129,2cyclictest32364-21uname19:35:180
1906199129,2cyclictest32364-21uname19:35:180
1906199129,2cyclictest30506-21head23:35:240
1906199129,2cyclictest30306-21basename21:40:120
1906199129,2cyclictest27628-21tr19:25:200
1906199129,2cyclictest23896-21chrt22:53:210
1906199129,2cyclictest22703-21cat20:25:000
1906199129,2cyclictest11738-21grep20:00:180
1906199128,3cyclictest20350-21tr23:10:180
1906199124,4cyclictest225-21systemd-journal21:05:010
19061991212,0cyclictest0-21swapper/023:18:240
1906199120,8cyclictest0-21swapper/000:24:570
1907199118,2cyclictest6383-21sed20:55:273
1907199118,2cyclictest30622-21head23:55:173
1907199118,2cyclictest23999-21ssh23:31:483
1907199110,10cyclictest14672-21tr20:05:233
1906799118,2cyclictest989-21ssh00:35:172
1906799118,2cyclictest9575-21load19:55:192
1906799118,2cyclictest8310-21cat22:25:112
1906799118,2cyclictest3436-21taskset23:20:102
1906799118,2cyclictest30267-21cut21:20:212
1906799118,2cyclictest30135-21expr23:55:142
1906799118,2cyclictest30039-21cut23:35:202
1906799118,2cyclictest27917-21head22:55:202
1906799118,2cyclictest25368-21ssh21:36:352
1906799118,2cyclictest19959-21ssh22:12:142
1906799118,2cyclictest17755-21ssh00:06:332
1906799118,2cyclictest1745-21cat19:40:122
1906799118,2cyclictest1642-21grep20:45:252
1906799118,2cyclictest12049-21grep20:00:202
1906799117,3cyclictest16406-21cat21:50:252
1906799115,2cyclictest225-21systemd-journal22:51:432
1906799111,6cyclictest0-21swapper/200:30:122
1906799110,1cyclictest0-21swapper/222:45:182
1906599118,2cyclictest7306-21ssh00:00:231
1906599118,2cyclictest4895-21ssh22:41:491
1906599118,2cyclictest438-21expr20:45:141
1906599118,2cyclictest31194-21tr20:40:201
1906599118,2cyclictest29660-21ssh22:37:221
1906599118,2cyclictest28401-21cut23:15:181
1906599118,2cyclictest27322-21head22:55:171
1906599118,2cyclictest26919-21rm21:37:481
1906599118,2cyclictest24176-21tr22:15:141
1906599118,2cyclictest20553-21ssh00:08:441
1906599118,2cyclictest19576-21grep19:10:141
1906599118,2cyclictest17972-21ssh23:09:321
1906599118,2cyclictest10492-21cat21:05:181
1906599116,2cyclictest225-21systemd-journal19:20:001
1906599115,5cyclictest23704-21tail20:25:151
1906599114,6cyclictest5205-21sensors_temp19:45:241
1906599113,1cyclictest9134-21runrttasks19:55:151
1906599111,9cyclictest0-21swapper/123:42:421
19065991110,1cyclictest0-21swapper/121:45:151
1906199119,1cyclictest7684-21ssh23:22:080
1906199118,2cyclictest6510-21tr21:45:160
1906199118,2cyclictest5917-21cat21:25:180
1906199118,2cyclictest464-21ssh22:00:230
1906199118,2cyclictest4575-21head19:45:170
1906199118,2cyclictest28991-21find20:35:230
1906199118,2cyclictest27965-21ssh00:31:540
1906199118,2cyclictest26129-21taskset19:23:420
1906199118,2cyclictest25918-21grep20:30:150
1906199118,2cyclictest23088-21grep20:25:130
1906199118,2cyclictest21724-21ssh21:35:090
1906199118,2cyclictest18407-21ssh21:51:510
1906199117,3cyclictest19305-21ls20:15:200
1906199113,7cyclictest0-21swapper/023:45:120
1906199110,10cyclictest3801-21ssh00:36:470
1906199110,0cyclictest0-21swapper/023:51:480
1907199108,1cyclictest777-21chrt19:38:113
1907199108,1cyclictest777-21chrt19:38:103
1907199108,1cyclictest5855-21chrt19:49:163
1907199108,1cyclictest28336-21taskset19:26:343
1907199108,1cyclictest19165-21sed20:15:183
1907199107,2cyclictest29426-21df19:30:163
1907199107,2cyclictest10469-21uname21:05:193
1906799108,1cyclictest32519-21cut19:35:212
1906799108,1cyclictest32519-21cut19:35:202
1906799108,1cyclictest15675-21chrt20:10:112
1906799107,2cyclictest6199-21aten2_r7power_c19:50:132
1906799107,2cyclictest3499-21uname20:50:192
1906799107,2cyclictest10351-21basename21:05:162
1906799102,2cyclictest0-21swapper/220:20:132
1906799100,9cyclictest23349-21unixbench_singl19:15:262
1906799100,9cyclictest0-21swapper/219:20:132
1906599108,1cyclictest20943-21expr20:20:151
1906599108,1cyclictest18219-21wget20:15:131
1906599107,2cyclictest6762-21sh19:50:151
1906599107,2cyclictest5905-21ntpq20:55:211
1906599107,2cyclictest28071-21cat20:35:151
1906599107,2cyclictest16496-21grep00:25:161
1906599100,9cyclictest26861-21expr19:25:141
1906199108,1cyclictest5859-21grep20:55:200
1906199107,2cyclictest724-21grep20:45:160
1906199107,2cyclictest6763-21df19:50:160
1906199107,2cyclictest4878-21cat23:40:130
1906199107,2cyclictest17466-21chrt20:10:370
1906199107,2cyclictest1681-21expr19:40:130
1906199100,9cyclictest2604-21ls20:50:140
190679997,1cyclictest4490-21fschecks_time19:45:172
190679996,2cyclictest13366-21grep20:05:122
190659997,1cyclictest16164-21expr20:10:151
190619997,1cyclictest14653-21grep20:05:210
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional