You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-13 - 01:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Sun Apr 12, 2026 12:45:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3092204997568,4cyclictest0-21swapper/010:00:010
309220499739,5cyclictest131rcu_preempt09:40:010
309220999693,4cyclictest131rcu_preempt09:25:011
309220499698,10cyclictest12-21ksoftirqd/009:30:000
309220499651,11cyclictest121ksoftirqd/007:40:000
309221599644,45cyclictest301ksoftirqd/207:40:022
3092204996327,7cyclictest12-21ksoftirqd/009:41:580
309220499621,9cyclictest1-21systemd11:45:000
3092204995940,12cyclictest3092196-21cyclictest07:10:020
309220499593,10cyclictest12-21ksoftirqd/007:25:000
3092204995915,7cyclictest12-21ksoftirqd/007:30:030
309220999585,3cyclictest131rcu_preempt08:15:011
3092204995820,8cyclictest12-21ksoftirqd/008:40:020
309221599576,6cyclictest131rcu_preempt10:20:002
309220499574,28cyclictest131rcu_preempt11:09:350
3092204995729,13cyclictest3108400-21ssh09:33:060
3092204995722,31cyclictest12-21ksoftirqd/010:54:330
3092204995722,31cyclictest12-21ksoftirqd/010:54:330
3092204995718,9cyclictest12-21ksoftirqd/012:05:020
309221599564,7cyclictest131rcu_preempt11:55:022
309221599564,7cyclictest131rcu_preempt11:55:022
309220499567,9cyclictest12-21ksoftirqd/008:05:010
3092204995649,5cyclictest19350irq/34-mmc107:35:020
309221599556,7cyclictest131rcu_preempt09:09:592
309220499558,42cyclictest3104876-21unin-run09:25:010
309220499558,10cyclictest12-21ksoftirqd/010:40:020
3092204995521,17cyclictest16850irq/134-800000009:15:080
3092204995519,6cyclictest16850irq/134-800000011:28:080
3092204995519,6cyclictest16850irq/134-800000011:28:080
3092204995519,18cyclictest16850irq/134-800000012:15:410
3092204995519,18cyclictest16850irq/134-800000012:15:400
3092204995511,5cyclictest131rcu_preempt09:14:220
309221599548,7cyclictest131rcu_preempt08:54:592
309221599546,5cyclictest131rcu_preempt11:28:232
309221599546,5cyclictest131rcu_preempt11:28:232
3092215995446,5cyclictest3136204-21kworker/2:311:46:102
3092215995446,5cyclictest3127220-21kworker/2:111:07:482
3092215995446,5cyclictest3094793-21kworker/2:210:34:522
3092215995444,7cyclictest3093267-21kworker/2:007:58:142
3092204995435,5cyclictest16850irq/134-800000010:56:040
3092204995428,12cyclictest3130817-21ssh11:15:190
3092204995428,12cyclictest3130817-21ssh11:15:190
3092204995424,15cyclictest3114675-21ssh09:47:370
3092204995424,15cyclictest3114675-21ssh09:47:370
3092204995420,19cyclictest16850irq/134-800000012:13:210
309221599536,7cyclictest30-21ksoftirqd/211:15:022
309221599536,6cyclictest131rcu_preempt10:45:002
3092215995345,5cyclictest3094793-21kworker/2:210:37:482
3092215995344,5cyclictest3094793-21kworker/2:210:45:572
3092209995340,10cyclictest0-21swapper/107:10:021
309220999531,3cyclictest131rcu_preempt11:40:021
309220499539,10cyclictest12-21ksoftirqd/010:50:020
309220499534,10cyclictest12-21ksoftirqd/010:15:020
309220499534,10cyclictest12-21ksoftirqd/010:15:020
3092204995325,12cyclictest3146985-21ssh12:27:550
3092204995318,18cyclictest16850irq/134-800000011:22:280
309221599528,12cyclictest30-21ksoftirqd/211:00:002
309221599526,10cyclictest30-21ksoftirqd/208:39:592
3092215995244,5cyclictest3127220-21kworker/2:111:40:352
3092215995244,5cyclictest3097393-21kworker/2:309:22:512
3092215995244,5cyclictest3094793-21kworker/2:211:16:332
3092215995244,5cyclictest3094793-21kworker/2:211:16:332
3092215995244,5cyclictest3094793-21kworker/2:209:34:382
3092215995243,6cyclictest3094793-21kworker/2:209:55:242
3092215995243,5cyclictest3097393-21kworker/2:309:41:422
3092215995243,5cyclictest3094793-21kworker/2:209:54:372
3092215995242,6cyclictest3093267-21kworker/2:008:10:002
309221599522,5cyclictest131rcu_preempt08:15:002
309220499525,7cyclictest12-21ksoftirqd/011:55:020
309220499525,7cyclictest12-21ksoftirqd/011:55:020
3092204995218,18cyclictest16850irq/134-800000011:31:030
3092204995215,16cyclictest0-21swapper/009:00:030
309221599519,6cyclictest131rcu_preempt10:54:192
309221599519,6cyclictest131rcu_preempt10:54:192
3092215995143,5cyclictest3127220-21kworker/2:112:38:112
3092215995143,5cyclictest3127220-21kworker/2:111:01:052
3092215995143,5cyclictest3094793-21kworker/2:209:49:442
3092215995143,5cyclictest3094793-21kworker/2:209:49:442
3092215995143,5cyclictest3094793-21kworker/2:209:36:392
3092215995143,5cyclictest3094793-21kworker/2:209:12:012
3092215995142,6cyclictest3127220-21kworker/2:111:35:122
3092204995133,13cyclictest3467602-21systemd-journal12:10:010
3092204995123,13cyclictest3116401-21ssh09:51:370
309220499511,8cyclictest3136151-21/usr/sbin/munin11:40:010
309220499511,16cyclictest12-21ksoftirqd/007:54:590
309221799509,3cyclictest131rcu_preempt10:15:013
309221799509,3cyclictest131rcu_preempt10:15:003
309221799508,6cyclictest131rcu_preempt10:40:003
3092217995033,13cyclictest1-21systemd10:10:013
309221799502,8cyclictest3136887-21ssh11:42:483
3092215995042,5cyclictest3139543-21kworker/2:212:17:452
3092215995042,5cyclictest3139543-21kworker/2:212:17:452
3092215995042,5cyclictest3139543-21kworker/2:212:14:042
3092215995042,5cyclictest3139543-21kworker/2:212:06:092
3092215995042,5cyclictest3136204-21kworker/2:311:56:502
3092215995042,5cyclictest3136204-21kworker/2:311:56:502
3092215995042,5cyclictest3127220-21kworker/2:111:31:212
3092215995042,5cyclictest3097393-21kworker/2:309:15:382
3092215995042,5cyclictest3094793-21kworker/2:210:02:342
3092209995033,13cyclictest275-21dbus-daemon12:30:011
309220999502,4cyclictest131rcu_preempt08:55:001
309220499505,10cyclictest12-21ksoftirqd/008:15:000
309220499503,14cyclictest12-21ksoftirqd/009:10:030
309220499502,14cyclictest12-21ksoftirqd/012:40:000
309221799498,7cyclictest37-21ksoftirqd/312:35:193
3092215994942,4cyclictest3094793-21kworker/2:210:20:022
3092215994941,5cyclictest3139543-21kworker/2:212:30:332
3092215994941,5cyclictest3127220-21kworker/2:112:28:282
3092215994941,5cyclictest3127220-21kworker/2:112:21:382
3092215994941,5cyclictest3097393-21kworker/2:309:25:142
3092215994941,5cyclictest3097393-21kworker/2:308:55:162
3092215994941,5cyclictest3094793-21kworker/2:211:20:352
3092215994941,5cyclictest3094793-21kworker/2:208:47:202
3092215994941,5cyclictest3094793-21kworker/2:208:40:042
3092215994941,5cyclictest3082899-21kworker/2:207:30:212
3092215994941,5cyclictest3082899-21kworker/2:207:17:422
3092215994941,5cyclictest3082899-21kworker/2:207:11:252
3092215994941,5cyclictest3082899-21kworker/2:207:11:242
309220499495,23cyclictest19350irq/34-mmc111:10:030
3092204994935,9cyclictest12-21ksoftirqd/008:00:010
309221799488,6cyclictest131rcu_preempt11:35:023
3092217994837,8cyclictest48-21kcompactd009:31:183
3092217994828,16cyclictest3467602-21systemd-journal11:05:003
3092217994811,6cyclictest131rcu_preempt07:10:023
309221599488,6cyclictest131rcu_preempt07:45:012
3092215994840,5cyclictest3093267-21kworker/2:007:52:122
3092215994839,6cyclictest3094793-21kworker/2:210:10:122
3092215994839,6cyclictest3094793-21kworker/2:210:10:122
3092215994839,6cyclictest3094793-21kworker/2:210:09:582
3092215994839,5cyclictest3127220-21kworker/2:112:00:522
3092209994833,11cyclictest3093731-21systemctl07:40:001
309220999482,4cyclictest131rcu_preempt12:35:021
309220999482,3cyclictest131rcu_preempt11:15:001
3092209994819,16cyclictest3092196-21cyclictest08:00:011
3092204994832,12cyclictest3121561-21systemd-run10:35:010
309220499483,10cyclictest12-21ksoftirqd/012:25:000
3092204994810,9cyclictest0-21swapper/009:03:190
3092215994740,4cyclictest3097393-21kworker/2:309:00:202
3092215994739,5cyclictest3094793-21kworker/2:210:25:202
3092215994739,5cyclictest3094793-21kworker/2:208:30:292
3092215994739,5cyclictest3094793-21kworker/2:208:26:072
3092215994739,5cyclictest3093267-21kworker/2:008:20:122
3092215994739,5cyclictest3093267-21kworker/2:008:20:122
3092215994739,5cyclictest3093267-21kworker/2:007:47:042
3092215994739,5cyclictest3082899-21kworker/2:207:25:452
309220999474,4cyclictest131rcu_preempt09:40:011
309220999474,3cyclictest131rcu_preempt11:30:011
309220999474,3cyclictest131rcu_preempt11:30:011
309220999474,3cyclictest131rcu_preempt09:15:021
3092209994731,12cyclictest3115575-21systemd-run09:50:011
3092209994731,12cyclictest3115575-21systemd-run09:50:011
3092209994729,15cyclictest3467602-21systemd-journal10:24:561
309220999472,3cyclictest131rcu_preempt09:20:011
309220499474,7cyclictest12-21ksoftirqd/011:49:190
309220499473,8cyclictest12-21ksoftirqd/010:00:290
309220499472,8cyclictest12-21ksoftirqd/008:20:000
309221799469,7cyclictest37-21ksoftirqd/312:25:003
309221799468,6cyclictest131rcu_preempt09:17:443
309221799462,13cyclictest3100404-21ssh09:14:323
309221799461,3cyclictest131rcu_preempt12:19:193
309221799461,3cyclictest131rcu_preempt12:19:193
3092215994639,4cyclictest3093267-21kworker/2:008:01:352
3092215994638,5cyclictest3082899-21kworker/2:207:21:262
309220999463,3cyclictest131rcu_preempt11:05:001
3092209994630,12cyclictest3467602-21systemd-journal07:24:171
309220999462,5cyclictest131rcu_preempt10:20:001
309220999461,4cyclictest131rcu_preempt09:42:191
309220499466,8cyclictest12-21ksoftirqd/011:01:110
3092204994633,9cyclictest3148170-21sshd12:33:190
309220499462,14cyclictest3467602-21systemd-journal10:43:290
3092204994611,5cyclictest131rcu_preempt08:50:010
309221799454,3cyclictest131rcu_preempt11:40:003
3092217994535,6cyclictest309-21in:imuxsock11:20:013
3092217994535,6cyclictest309-21in:imuxsock11:20:013
309221799453,4cyclictest131rcu_preempt11:00:003
3092217994527,14cyclictest1-21systemd11:55:013
3092217994527,14cyclictest1-21systemd11:55:013
309221799452,39cyclictest275-21dbus-daemon08:10:013
3092217994512,5cyclictest131rcu_preempt07:12:173
3092217994512,5cyclictest131rcu_preempt07:12:163
3092215994537,5cyclictest3093267-21kworker/2:008:15:032
309220999455,4cyclictest131rcu_preempt10:00:021
309220999452,4cyclictest131rcu_preempt12:25:001
309220999451,3cyclictest131rcu_preempt11:07:191
309220499455,9cyclictest12-21ksoftirqd/011:56:550
309220499455,9cyclictest12-21ksoftirqd/011:56:550
309220499455,13cyclictest12-21ksoftirqd/007:12:170
309220499455,13cyclictest12-21ksoftirqd/007:12:160
3092204994513,22cyclictest151rcuc/010:07:200
3092204994512,5cyclictest12-21ksoftirqd/007:18:330
3092217994432,9cyclictest37-21ksoftirqd/311:15:003
3092217994427,14cyclictest275-21dbus-daemon07:25:013
309220999445,23cyclictest3141822-21ssh12:05:011
309220999443,3cyclictest131rcu_preempt08:10:011
309220999442,4cyclictest131rcu_preempt11:00:001
309220999442,3cyclictest131rcu_preempt10:05:001
309220999441,4cyclictest131rcu_preempt12:00:001
309220999441,4cyclictest131rcu_preempt12:00:001
309220999441,3cyclictest131rcu_preempt10:40:001
309220999441,3cyclictest131rcu_preempt08:45:001
309220499445,8cyclictest12-21ksoftirqd/007:45:020
309220499444,33cyclictest0-21swapper/008:30:010
309220499443,11cyclictest3094231-21sshd07:49:260
309220499442,8cyclictest12-21ksoftirqd/010:25:020
3092204994428,13cyclictest3097086-21sshd08:42:190
309220499441,5cyclictest0-21swapper/008:10:010
309221799439,5cyclictest131rcu_preempt09:25:553
309221799433,5cyclictest3114081-21ssh09:46:113
309221799433,5cyclictest3114081-21ssh09:46:103
309221799432,10cyclictest3095781-21sshd08:19:313
309221799431,6cyclictest0-21swapper/312:09:593
309221799431,4cyclictest131rcu_preempt09:21:303
309220999435,11cyclictest131rcu_preempt10:15:001
309220999435,11cyclictest131rcu_preempt10:15:001
309220999434,4cyclictest131rcu_preempt12:10:011
309220999432,4cyclictest131rcu_preempt09:10:021
309220999432,3cyclictest131rcu_preempt09:00:011
309220999431,3cyclictest131rcu_preempt11:49:481
309221799427,6cyclictest131rcu_preempt10:45:013
309221799425,3cyclictest131rcu_preempt07:35:013
309221799423,6cyclictest131rcu_preempt09:05:023
309221799423,5cyclictest131rcu_preempt11:47:033
3092217994230,9cyclictest37-21ksoftirqd/308:55:023
3092217994225,13cyclictest275-21dbus-daemon07:55:013
309220999422,3cyclictest131rcu_preempt12:13:191
309220999422,3cyclictest131rcu_preempt11:20:021
309220999422,3cyclictest131rcu_preempt11:20:021
309220999422,3cyclictest131rcu_preempt09:30:391
3092209994218,9cyclictest23-21ksoftirqd/110:10:021
309220999421,4cyclictest131rcu_preempt08:50:011
309220999421,3cyclictest131rcu_preempt10:47:291
309220499425,7cyclictest12-21ksoftirqd/010:30:010
3092204994227,11cyclictest12-21ksoftirqd/010:19:590
309221799418,5cyclictest131rcu_preempt12:02:563
309221799414,8cyclictest37-21ksoftirqd/308:10:233
3092217994127,11cyclictest37-21ksoftirqd/312:15:003
3092217994125,12cyclictest3124904-21cron10:49:593
309221799411,7cyclictest0-21swapper/309:54:013
309221599412,6cyclictest131rcu_preempt07:10:022
309220999412,4cyclictest131rcu_preempt12:17:161
309220999412,4cyclictest131rcu_preempt12:17:161
309220999412,4cyclictest131rcu_preempt09:52:041
309220999412,3cyclictest131rcu_preempt10:35:021
309220999411,4cyclictest131rcu_preempt11:55:011
309220999411,4cyclictest131rcu_preempt11:55:011
309220999411,4cyclictest131rcu_preempt10:53:541
309220999411,4cyclictest131rcu_preempt10:53:541
309220999411,3cyclictest131rcu_preempt12:40:011
3092204994126,12cyclictest3467602-21systemd-journal08:25:010
3092204994126,12cyclictest3467602-21systemd-journal08:25:010
309220499411,9cyclictest3097695-21sed08:55:020
309221799407,6cyclictest131rcu_preempt09:41:173
309221799403,33cyclictest150-21pr/ttyS211:06:443
309221799401,8cyclictest131rcu_preempt09:37:413
309221799401,7cyclictest0-21swapper/310:51:383
309221799401,7cyclictest0-21swapper/310:51:383
309221799401,3cyclictest131rcu_preempt08:05:023
3092209994032,5cyclictest3095786-21cron08:20:001
309220999402,3cyclictest131rcu_preempt09:25:011
3092209994017,9cyclictest23-21ksoftirqd/108:05:021
309220999401,4cyclictest131rcu_preempt11:43:451
309220999401,3cyclictest131rcu_preempt11:34:121
309220999401,3cyclictest131rcu_preempt07:35:001
309220999401,3cyclictest131rcu_preempt07:30:031
309221799398,3cyclictest131rcu_preempt12:00:003
309221799398,3cyclictest131rcu_preempt12:00:003
309221799397,4cyclictest131rcu_preempt12:27:203
309221799396,6cyclictest131rcu_preempt10:31:343
309221799394,7cyclictest37-21ksoftirqd/308:50:023
309221799393,3cyclictest131rcu_preempt09:10:033
3092217993929,6cyclictest3133353-21sh11:26:563
3092217993929,6cyclictest3133353-21sh11:26:563
3092217993927,6cyclictest309-21in:imuxsock08:40:013
3092217993923,13cyclictest3119300-21sed10:05:013
309221799391,7cyclictest0-21swapper/312:32:113
309221799391,6cyclictest131rcu_preempt07:20:013
309220999393,3cyclictest131rcu_preempt07:55:011
309220999392,4cyclictest131rcu_preempt10:45:001
309220999391,4cyclictest131rcu_preempt11:25:001
309220999391,4cyclictest131rcu_preempt07:18:211
309220999391,3cyclictest131rcu_preempt08:40:021
309221799387,5cyclictest131rcu_preempt11:20:193
309221799387,3cyclictest131rcu_preempt08:25:013
309221799387,3cyclictest131rcu_preempt08:25:013
309221799386,3cyclictest131rcu_preempt08:45:013
309221799381,5cyclictest0-21swapper/309:55:373
3092209993828,7cyclictest3096710-21sed08:35:011
3092209993827,8cyclictest3098163-21pluginstate09:05:011
3092209993824,11cyclictest23-21ksoftirqd/110:25:091
309220999381,5cyclictest131rcu_preempt07:10:171
309220999381,5cyclictest131rcu_preempt07:10:171
309220999381,3cyclictest131rcu_preempt08:25:171
309221799375,3cyclictest131rcu_preempt08:29:323
309221799374,7cyclictest37-21ksoftirqd/308:58:503
3092217993724,10cyclictest37-21ksoftirqd/307:45:013
309220999374,3cyclictest131rcu_preempt07:45:021
309220999372,3cyclictest131rcu_preempt08:25:011
309220999372,3cyclictest131rcu_preempt08:25:011
3092204993714,5cyclictest131rcu_preempt08:35:010
3092217993622,9cyclictest3467602-21systemd-journal07:40:003
3092217993622,10cyclictest309-21in:imuxsock10:20:023
3092217993618,14cyclictest275-21dbus-daemon10:30:013
3092217993523,10cyclictest37-21ksoftirqd/307:30:013
3092217993522,9cyclictest3094742-21cron08:00:003
3092217993426,5cyclictest3120439-21latency_hist10:20:013
3092217993424,7cyclictest37-21ksoftirqd/307:50:013
3092217993422,9cyclictest37-21ksoftirqd/308:30:073
309220999341,3cyclictest131rcu_preempt07:50:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional